<?xml version="1.0" encoding="utf-8" ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="Docutils 0.7: http://docutils.sourceforge.net/" />
<title>PyInstaller Manual</title>
<meta name="author" content="Giovanni Bajo &amp; William Caban (based on Gordon McMillan's manual)" />
<meta name="copyright" content="This document has been placed in the public domain." />
<link rel="stylesheet" href="stylesheets/default.css" type="text/css" />
</head>
<body>
<div class="document" id="pyinstaller-manual">
<h1 class="title">PyInstaller Manual</h1>
<table class="docinfo" frame="void" rules="none">
<col class="docinfo-name" />
<col class="docinfo-content" />
<tbody valign="top">
<tr><th class="docinfo-name">Version:</th>
<td>PyInstaller 2.0</td></tr>
<tr class="field"><th class="docinfo-name">Homepage:</th><td class="field-body"><a class="reference external" href="http://www.pyinstaller.org">http://www.pyinstaller.org</a></td>
</tr>
<tr><th class="docinfo-name">Author:</th>
<td>Giovanni Bajo &amp; William Caban (based on Gordon McMillan's manual)</td></tr>
<tr><th class="docinfo-name">Contact:</th>
<td><a class="first last reference external" href="mailto:rasky&#64;develer.com">rasky&#64;develer.com</a></td></tr>
<tr><th class="docinfo-name">Revision:</th>
<td>$Rev$</td></tr>
<tr class="field"><th class="docinfo-name">Source URL:</th><td class="field-body">$HeadURL$</td>
</tr>
<tr><th class="docinfo-name">Copyright:</th>
<td>This document has been placed in the public domain.</td></tr>
</tbody>
</table>
<div class="contents topic" id="contents">
<p class="topic-title first">Contents</p>
<ul class="simple">
<li><a class="reference internal" href="#requirements" id="id1">Requirements</a></li>
<li><a class="reference internal" href="#installing-pyinstaller" id="id2">Installing <cite>PyInstaller</cite></a></li>
<li><a class="reference internal" href="#getting-started" id="id3">Getting Started</a><ul>
<li><a class="reference internal" href="#build-your-project" id="id4">Build your project</a></li>
<li><a class="reference internal" href="#allowed-options" id="id5">Allowed Options</a></li>
<li><a class="reference internal" href="#a-spec-file-for-your-project" id="id6">A spec file for your project</a></li>
<li><a class="reference internal" href="#windows-com-server-support" id="id7">Windows COM Server support</a></li>
<li><a class="reference internal" href="#building-optimized" id="id8">Building Optimized</a></li>
<li><a class="reference internal" href="#a-note-on-using-upx" id="id9">A Note on using UPX</a></li>
<li><a class="reference internal" href="#accessing-data-files" id="id10">Accessing Data Files</a><ul>
<li><a class="reference internal" href="#adopt-your-application" id="id11">Adopt your application</a></li>
<li><a class="reference internal" href="#collect-your-data-files" id="id12">Collect your data files</a></li>
</ul>
</li>
<li><a class="reference internal" href="#how-one-file-mode-works" id="id13">How one-file mode works</a></li>
<li><a class="reference internal" href="#egg-files-and-setuptools" id="id14">.egg files and setuptools</a></li>
<li><a class="reference internal" href="#multipackage-function" id="id15">Multipackage function</a></li>
</ul>
</li>
<li><a class="reference internal" href="#pyinstaller-utilities" id="id16">PyInstaller Utilities</a><ul>
<li><a class="reference internal" href="#archiveviewer" id="id17">ArchiveViewer</a></li>
<li><a class="reference internal" href="#bindepend" id="id18">BinDepend</a></li>
<li><a class="reference internal" href="#grabversion-windows" id="id19">GrabVersion (Windows)</a></li>
<li><a class="reference internal" href="#analyzing-dependencies" id="id20">Analyzing Dependencies</a></li>
</ul>
</li>
<li><a class="reference internal" href="#spec-files" id="id21">Spec Files</a><ul>
<li><a class="reference internal" href="#introduction" id="id22">Introduction</a></li>
<li><a class="reference internal" href="#toc-class-table-of-contents" id="id23">TOC Class (Table of Contents)</a></li>
<li><a class="reference internal" href="#target-subclasses" id="id24">Target Subclasses</a><ul>
<li><a class="reference internal" href="#analysis" id="id25">Analysis</a></li>
<li><a class="reference internal" href="#pyz" id="id26">PYZ</a></li>
<li><a class="reference internal" href="#pkg" id="id27">PKG</a></li>
<li><a class="reference internal" href="#exe" id="id28">EXE</a></li>
<li><a class="reference internal" href="#dll" id="id29">DLL</a></li>
<li><a class="reference internal" href="#collect" id="id30">COLLECT</a></li>
<li><a class="reference internal" href="#tree" id="id31">Tree</a></li>
<li><a class="reference internal" href="#merge" id="id32">MERGE</a></li>
</ul>
</li>
</ul>
</li>
<li><a class="reference internal" href="#when-things-go-wrong" id="id33">When Things Go Wrong</a><ul>
<li><a class="reference internal" href="#recipes-and-instructions-for-special-modules" id="id34">Recipes and Instructions for special Modules</a></li>
<li><a class="reference internal" href="#finding-out-what-went-wrong" id="id35">Finding out What Went Wrong</a><ul>
<li><a class="reference internal" href="#buildtime-warnings" id="id36">Buildtime Warnings</a></li>
<li><a class="reference internal" href="#getting-debug-messages" id="id37">Getting Debug Messages</a></li>
<li><a class="reference internal" href="#getting-python-s-verbose-imports" id="id38">Getting Python's Verbose Imports</a></li>
</ul>
</li>
<li><a class="reference internal" href="#helping-pyinstaller-find-modules" id="id39">Helping PyInstaller Find Modules</a><ul>
<li><a class="reference internal" href="#extending-the-path" id="id40">Extending the Path</a></li>
<li><a class="reference internal" href="#listing-hidden-imports" id="id41">Listing Hidden Imports</a></li>
<li><a class="reference internal" href="#extending-a-package-s-path" id="id42">Extending a Package's <tt class="docutils literal">__path__</tt></a></li>
<li><a class="reference internal" href="#changing-runtime-behavior" id="id43">Changing Runtime Behavior</a></li>
<li><a class="reference internal" href="#adapting-to-being-frozen" id="id44">Adapting to being &quot;frozen&quot;</a></li>
</ul>
</li>
</ul>
</li>
<li><a class="reference internal" href="#miscellaneous" id="id45">Miscellaneous</a><ul>
<li><a class="reference internal" href="#self-extracting-executables" id="id46">Self-extracting executables</a><ul>
<li><a class="reference internal" href="#one-pass-execution" id="id47">One Pass Execution</a></li>
<li><a class="reference internal" href="#two-pass-execution" id="id48">Two Pass Execution</a></li>
</ul>
</li>
</ul>
</li>
<li><a class="reference internal" href="#pyinstaller-archives" id="id49">PyInstaller Archives</a><ul>
<li><a class="reference internal" href="#archives-introduction" id="id50">Archives Introduction</a></li>
<li><a class="reference internal" href="#zlibarchive" id="id51"><tt class="docutils literal">ZlibArchive</tt></a></li>
<li><a class="reference internal" href="#carchive" id="id52"><tt class="docutils literal">CArchive</tt></a></li>
</ul>
</li>
<li><a class="reference internal" href="#license" id="id53">License</a></li>
<li><a class="reference internal" href="#appendix" id="id54">Appendix</a><ul>
<li><a class="reference internal" href="#building-the-bootloaders" id="id55">Building the bootloaders</a><ul>
<li><a class="reference internal" href="#development-tools" id="id56">Development tools</a></li>
<li><a class="reference internal" href="#building" id="id57">Building</a></li>
<li><a class="reference internal" href="#linux-standard-base-lsb-binary" id="id58">Linux Standard Base (LSB) binary</a></li>
</ul>
</li>
<li><a class="reference internal" href="#mf-py-a-modulefinder-replacement" id="id59"><tt class="docutils literal">mf.py</tt>: A Modulefinder Replacement</a><ul>
<li><a class="reference internal" href="#importtracker" id="id60">ImportTracker</a></li>
<li><a class="reference internal" href="#analyze-one" id="id61"><tt class="docutils literal">analyze_one()</tt></a></li>
<li><a class="reference internal" href="#module-classes" id="id62">Module Classes</a></li>
<li><a class="reference internal" href="#code-scanning" id="id63">code scanning</a></li>
<li><a class="reference internal" href="#hooks" id="id64">Hooks</a></li>
</ul>
</li>
<li><a class="reference internal" href="#advanced-hook-usage" id="id65">Advanced Hook Usage</a><ul>
<li><a class="reference internal" href="#warnings" id="id66">Warnings</a></li>
<li><a class="reference internal" href="#cross-reference" id="id67">Cross Reference</a></li>
<li><a class="reference internal" href="#mf-usage" id="id68">mf Usage</a></li>
</ul>
</li>
<li><a class="reference internal" href="#iu-py-an-imputil-replacement" id="id69"><tt class="docutils literal">iu.py</tt>: An <em>imputil</em> Replacement</a><ul>
<li><a class="reference internal" href="#importmanager" id="id70"><tt class="docutils literal">ImportManager</tt></a></li>
<li><a class="reference internal" href="#importdirector" id="id71"><tt class="docutils literal">ImportDirector</tt></a></li>
<li><a class="reference internal" href="#pathimportdirector" id="id72"><tt class="docutils literal">PathImportDirector</tt></a></li>
<li><a class="reference internal" href="#owner" id="id73"><tt class="docutils literal">Owner</tt></a></li>
<li><a class="reference internal" href="#packages" id="id74">Packages</a></li>
<li><a class="reference internal" href="#possibilities" id="id75">Possibilities</a></li>
<li><a class="reference internal" href="#compatibility" id="id76">Compatibility</a></li>
<li><a class="reference internal" href="#performance" id="id77">Performance</a></li>
<li><a class="reference internal" href="#limitations" id="id78">Limitations</a></li>
<li><a class="reference internal" href="#iu-usage" id="id79">iu Usage</a></li>
</ul>
</li>
</ul>
</li>
</ul>
</div>
<div class="section" id="requirements">
<h1><a class="toc-backref" href="#id1">Requirements</a></h1>
<dl class="docutils">
<dt><strong>Windows</strong></dt>
<dd><ul class="first last simple">
<li>Windows XP or newer.</li>
<li><a class="reference external" href="http://sourceforge.net/projects/pywin32/files/">PyWin32</a>
Python extensions for Windows is only necessary for users of Python 2.6+.</li>
</ul>
</dd>
<dt><strong>Linux</strong></dt>
<dd><ul class="first last simple">
<li>ldd
- Console application to print the shared libraries required by each program
or shared library.</li>
<li>objdump
- Console application to display information from object files.</li>
</ul>
</dd>
<dt><strong>Mac OS X</strong></dt>
<dd><ul class="first last simple">
<li>Mac OS X 10.4 (Tiger) or newer (Leopard, Snow Leopard, Lion, Mountain Lion).</li>
</ul>
</dd>
<dt><strong>Solaris</strong></dt>
<dd><ul class="first last simple">
<li>ldd</li>
<li>objdump</li>
</ul>
</dd>
<dt><strong>AIX</strong></dt>
<dd><ul class="first last simple">
<li>AIX 6.1 or newer.
Python executables created using PyInstaller on AIX 6.1 should work
on AIX 5.2/5.3. PyInstaller will not work with statically linked Python
libraries which has been encountered in Python 2.2 installations on AIX 5.x.</li>
<li>ldd</li>
<li>objdump</li>
</ul>
</dd>
</dl>
</div>
<div class="section" id="installing-pyinstaller">
<h1><a class="toc-backref" href="#id2">Installing <cite>PyInstaller</cite></a></h1>
<ol class="arabic">
<li><p class="first">Unpack the archive on you path of choice. For the purpose of this
documentation we will assume /your/path/to/pyinstaller/.</p>
<p>You will be using a couple of scripts in the /your/path/to/pyinstaller/
directory, and these will find everything they need from their own
location. For convenience, keep the paths to these scripts short
(don't install in a deeply nested subdirectory).</p>
<p><strong>Please note:</strong> Installer is <cite>not</cite> a Python package, so it
doesn't need to go in site-packages, or have a .pth file.</p>
</li>
<li><p class="first">For Windows (32/64bit), Linux (32/64bit) and Mac OS X (32/64bit)
precompiled boot-loaders are available. So the installation is
complete now.</p>
<p>For other platforms (Solaris, AIX, etc.), users should first try to build the
boot-loader:</p>
<blockquote>
<p>cd source
python ./waf configure build install</p>
</blockquote>
</li>
</ol>
</div>
<div class="section" id="getting-started">
<h1><a class="toc-backref" href="#id3">Getting Started</a></h1>
<p>For the purpose of this documentation we will assume <cite>PyInstaller</cite> as
installed into /your/path/to/pyinstaller/.</p>
<div class="section" id="build-your-project">
<h2><a class="toc-backref" href="#id4">Build your project</a></h2>
<p>For building a Windows COM server, please see section <a class="reference internal" href="#windows-com-server-support">Windows COM
Server Support</a> below.</p>
<p>In the /your/path/to/pyinstaller/ directory, run:</p>
<pre class="literal-block">
python pyinstaller.py [opts] yourprogram.py
</pre>
<p>This will create a sub-directory <tt class="docutils literal"><span class="pre">your-program</span></tt> in the /your/path/to/pyinstaller/
directory. The generated files will be placed within the sub-directory
<tt class="docutils literal"><span class="pre">your-program/dist</span></tt>; that's where the files you are interested in
will be placed. A <cite>spec</cite> file called <tt class="docutils literal"><span class="pre">your-program.spec</span></tt> will be
created in the sub-directory <tt class="docutils literal"><span class="pre">your-program</span></tt>, too. Additionally a
subtracts <tt class="docutils literal"><span class="pre">your-program/build</span></tt> is created where intermediate build
files are kept.</p>
<p>If your current working directory is not /your/path/to/pyinstaller/, the
directories <tt class="docutils literal">dist</tt> and <tt class="docutils literal">build</tt> and the <cite>spec</cite> file will be
created in the current working directory. Say: the intermediate
directory <tt class="docutils literal"><span class="pre">your-program</span></tt> will be skipped.</p>
<p>If you have already created a <cite>spec</cite> file for your project then in
the /your/path/to/pyinstaller/ directory run:</p>
<pre class="literal-block">
python pyinstaller.py [opts] your-program.spec
</pre>
<p>If your current working directory is not /your/path/to/pyinstaller/, this works
analogously.</p>
<p>If everything is working and you are happy with the default settings,
this will be all you have to do. If not, see <a class="reference internal" href="#allowed-options">Allowed OPTIONS</a>, <a class="reference internal" href="#when-things-go-wrong">When
things go wrong</a> and be sure to read the introduction to <a class="reference internal" href="#spec-files">Spec
Files</a>.</p>
</div>
<div class="section" id="allowed-options">
<h2><a class="toc-backref" href="#id5">Allowed Options</a></h2>
<p>By default, <tt class="docutils literal">pyinstaller.py</tt> creates a distribution directory containing the main
executable and the dynamic libraries. The option <tt class="docutils literal"><span class="pre">--onefile</span></tt> (specifies that you want
PyInstaller to build a single file with everything inside.</p>
<p>The syntax to use <tt class="docutils literal">pyinstaller.py</tt> is the following:</p>
<pre class="literal-block">
python pyinstaller.py [opts] &lt;scriptname&gt; [ &lt;scriptname&gt; ...] | &lt;specfile&gt;
</pre>
<p>Allowed OPTIONS are:</p>
<table class="docutils option-list" frame="void" rules="none">
<col class="option" />
<col class="description" />
<tbody valign="top">
<tr><td class="option-group">
<kbd><span class="option">-h</span>, <span class="option">--help</span></kbd></td>
<td>show this help message and exit</td></tr>
<tr><td class="option-group">
<kbd><span class="option">-v</span>, <span class="option">--version</span></kbd></td>
<td>show program version</td></tr>
<tr><td class="option-group" colspan="2">
<kbd><span class="option">--upx-dir=<var>UPX_DIR</var></span></kbd></td>
</tr>
<tr><td>&nbsp;</td><td>Directory containing UPX.</td></tr>
<tr><td class="option-group">
<kbd><span class="option">-a</span>, <span class="option">--ascii</span></kbd></td>
<td>do NOT include unicode encodings (default: included if
available)</td></tr>
<tr><td class="option-group" colspan="2">
<kbd><span class="option">--buildpath=<var>BUILDPATH</var></span></kbd></td>
</tr>
<tr><td>&nbsp;</td><td>Buildpath (default:
SPECPATH/build/pyi.TARGET_PLATFORM/SPECNAME)</td></tr>
<tr><td class="option-group" colspan="2">
<kbd><span class="option">-y</span>, <span class="option">--noconfirm</span></kbd></td>
</tr>
<tr><td>&nbsp;</td><td>Remove output directory (default:
SPECPATH/dist/SPECNAME) without confirmation</td></tr>
<tr><td class="option-group" colspan="2">
<kbd><span class="option">--log-level=<var>LOGLEVEL</var></span></kbd></td>
</tr>
<tr><td>&nbsp;</td><td>Log level (default: INFO, choose one of DEBUG, INFO,
WARN, ERROR, CRITICAL)</td></tr>
</tbody>
</table>
<p>What to generate:</p>
<table class="docutils option-list" frame="void" rules="none">
<col class="option" />
<col class="description" />
<tbody valign="top">
<tr><td class="option-group">
<kbd><span class="option">-F</span>, <span class="option">--onefile</span></kbd></td>
<td>create a single file deployment</td></tr>
<tr><td class="option-group">
<kbd><span class="option">-D</span>, <span class="option">--onedir</span></kbd></td>
<td>create a single directory deployment (default)</td></tr>
<tr><td class="option-group" colspan="2">
<kbd><span class="option">-o <var>DIR</var></span>, <span class="option">--out=<var>DIR</var></span></kbd></td>
</tr>
<tr><td>&nbsp;</td><td>create the spec file in <em>directory</em>. If not specified, and the current
directory is Installer's root directory, an output subdirectory will be
created. Otherwise the current directory is used.</td></tr>
<tr><td class="option-group" colspan="2">
<kbd><span class="option">-n <var>NAME</var></span>, <span class="option">--name=<var>NAME</var></span></kbd></td>
</tr>
<tr><td>&nbsp;</td><td>optional <em>name</em> to assign to the project (from which the spec file name is
generated). If omitted, the basename of the (first) script is used.</td></tr>
</tbody>
</table>
<p>What to bundle, where to search:</p>
<table class="docutils option-list" frame="void" rules="none">
<col class="option" />
<col class="description" />
<tbody valign="top">
<tr><td class="option-group" colspan="2">
<kbd><span class="option">-p <var>DIR</var></span>, <span class="option">--paths=<var>DIR</var></span></kbd></td>
</tr>
<tr><td>&nbsp;</td><td>set base path for import (like using PYTHONPATH). Multiple directories are
allowed, separating them with the path separator (';' under Windows, ':'
under Linux), or using this option multiple times.</td></tr>
<tr><td class="option-group" colspan="2">
<kbd><span class="option">--hidden-import=<var>MODULENAME</var></span></kbd></td>
</tr>
<tr><td>&nbsp;</td><td>import hidden in the script(s). This option can be
used multiple times.</td></tr>
<tr><td class="option-group" colspan="2">
<kbd><span class="option">--additional-hooks-dir=<var>HOOKSPATH</var></span></kbd></td>
</tr>
<tr><td>&nbsp;</td><td>Additional path to search for hooks. This will go into the extend
the <cite>hookspath</cite>, see <a class="reference internal" href="#analysis">Analysis</a> below. This option may be given
several times.</td></tr>
</tbody>
</table>
<p>How to generate:</p>
<table class="docutils option-list" frame="void" rules="none">
<col class="option" />
<col class="description" />
<tbody valign="top">
<tr><td class="option-group">
<kbd><span class="option">-d</span>, <span class="option">--debug</span></kbd></td>
<td>use the debug (verbose) build of the executable</td></tr>
<tr><td class="option-group">
<kbd><span class="option">-s</span>, <span class="option">--strip</span></kbd></td>
<td>the executable and all shared libraries will be run through strip. Note
that cygwin's strip tends to render normal Win32 dlls unusable.</td></tr>
<tr><td class="option-group">
<kbd><span class="option">--noupx</span></kbd></td>
<td>do not use UPX even if available (works differently
between Windows and *nix)</td></tr>
</tbody>
</table>
<p>Windows and Mac OS X specific options:</p>
<table class="docutils option-list" frame="void" rules="none">
<col class="option" />
<col class="description" />
<tbody valign="top">
<tr><td class="option-group" colspan="2">
<kbd><span class="option">-c</span>, <span class="option">--console</span>, <span class="option">--nowindowed</span></kbd></td>
</tr>
<tr><td>&nbsp;</td><td>use a console subsystem executable (default)</td></tr>
<tr><td class="option-group" colspan="2">
<kbd><span class="option">-w</span>, <span class="option">--windowed</span>, <span class="option">--noconsole</span></kbd></td>
</tr>
<tr><td>&nbsp;</td><td>use a windowed subsystem executable, which on Windows
does not open the console when the program is launched.
On Mac OS X it allows running gui applications and also
creates also .app bundle.
<strong>This option is mandatory when freezing an gui application on Mac OS X.
Otherwise the application will not start.</strong>.</td></tr>
</tbody>
</table>
<dl class="docutils">
<dt>-i FILE.ICO, -i FILE.EXE,ID, -i FILE.ICNS, --icon=FILE.ICO, --icon=FILE.EXE,ID, --icon=FILE.ICNS</dt>
<dd>If FILE is an .ico file, add the icon to the final
executable. Otherwise, the syntax 'file.exe,id' to
extract the icon with the specified id from file.exe
and add it to the final executable. If FILE is an
.icns file, add the icon to the final .app bundle on
Mac OS X (for Mac not yet implemented)</dd>
</dl>
<p>Windows specific options:</p>
<table class="docutils option-list" frame="void" rules="none">
<col class="option" />
<col class="description" />
<tbody valign="top">
<tr><td class="option-group" colspan="2">
<kbd><span class="option">--version-file=<var>FILE</var></span></kbd></td>
</tr>
<tr><td>&nbsp;</td><td>add a version resource from FILE to the exe</td></tr>
<tr><td class="option-group" colspan="2">
<kbd><span class="option">-m <var>FILE</var></span>, <span class="option">-m <var>XML</var></span>, <span class="option">--manifest=<var>FILE</var></span>, <span class="option">--manifest=<var>XML</var></span></kbd></td>
</tr>
<tr><td>&nbsp;</td><td>add manifest FILE or XML to the exe</td></tr>
</tbody>
</table>
<dl class="docutils">
<dt>-r FILE[,TYPE[,NAME[,LANGUAGE]]], --resource=FILE[,TYPE[,NAME[,LANGUAGE]]]</dt>
<dd>add/update resource of the given type, name and
language from FILE to the final executable. FILE can
be a data file or an exe/dll. For data files, atleast
TYPE and NAME need to be specified, LANGUAGE defaults
to 0 or may be specified as wildcard * to update all
resources of the given TYPE and NAME. For exe/dll
files, all resources from FILE will be added/updated
to the final executable if TYPE, NAME and LANGUAGE are
omitted or specified as wildcard *.Multiple resources
are allowed, using this option multiple times.</dd>
</dl>
<p>For building with optimization on (like <tt class="docutils literal">Python <span class="pre">-O</span></tt>), see section
<a class="reference internal" href="#building-optimized">Building Optimized</a>.</p>
</div>
<div class="section" id="a-spec-file-for-your-project">
<h2><a class="toc-backref" href="#id6">A spec file for your project</a></h2>
<p>The spec file is the description of what you want <cite>PyInstaller</cite> to do with
your program. By deafult, <tt class="docutils literal">pyinstaller.py</tt> generates a spec file automatically.
For simple projects, the generated spec file will be probably sufficient.</p>
<p>For more complex projects, it should be regarded as a template. The spec file is
actually Python code, and modifying it should be ease. See <a class="reference internal" href="#spec-files">Spec Files</a> for
details.</p>
<p>In the root directory of <cite>PyInstaller</cite>, there is a simple wizard to create simple
spec files that cover all basic usages:</p>
<pre class="literal-block">
python utils/Makespec.py [--onefile] yourprogram.py
</pre>
<p>Elaborating on Makespec.py, this is the supported command line:</p>
<pre class="literal-block">
python utils/Makespec.py [opts] &lt;scriptname&gt; [&lt;scriptname&gt; ...]
</pre>
<p>Script <tt class="docutils literal">Makespec.py</tt> shares some options with <tt class="docutils literal">pyinstaller.py</tt>. For allowed options see:</p>
<pre class="literal-block">
python utils/Makespec.py --help
</pre>
</div>
<div class="section" id="windows-com-server-support">
<h2><a class="toc-backref" href="#id7">Windows COM Server support</a></h2>
<p>For Windows COM support execute:</p>
<pre class="literal-block">
python MakeComServer.py [OPTION] script...
</pre>
<p>This will generate a new script <tt class="docutils literal">drivescript.py</tt> and a spec file for the script.</p>
<p>These options are allowed:</p>
<table class="docutils option-list" frame="void" rules="none">
<col class="option" />
<col class="description" />
<tbody valign="top">
<tr><td class="option-group">
<kbd><span class="option">--debug</span></kbd></td>
<td>Use the verbose version of the executable.</td></tr>
<tr><td class="option-group">
<kbd><span class="option">--verbose</span></kbd></td>
<td>Register the COM server(s) with the quiet flag off.</td></tr>
<tr><td class="option-group">
<kbd><span class="option">--ascii</span></kbd></td>
<td>do not include encodings (this is passed through to Makespec).</td></tr>
<tr><td class="option-group">
<kbd><span class="option">--out <var>&lt;dir&gt;</var></span></kbd></td>
<td>Generate the driver script and spec file in dir.</td></tr>
</tbody>
</table>
<p>Now <a class="reference internal" href="#build-your-project">Build your project</a> on the generated spec file.</p>
<p>If you have the win32dbg package installed, you can use it with the generated
COM server. In the driver script, set <tt class="docutils literal">debug=1</tt> in the registration line.</p>
<p><strong>Warnings</strong>: the inprocess COM server support will not work when the client
process already has Python loaded. It would be rather tricky to
non-obtrusively hook into an already running Python, but the show-stopper is
that the Python/C API won't let us find out which interpreter instance I should
hook into. (If this is important to you, you might experiment with using
apartment threading, which seems the best possibility to get this to work). To
use a &quot;frozen&quot; COM server from a Python process, you'll have to load it as an
exe:</p>
<pre class="literal-block">
o = win32com.client.Dispatch(progid,
                 clsctx=pythoncom.CLSCTX_LOCAL_SERVER)
</pre>
<p>MakeCOMServer also assumes that your top level code (registration etc.) is
&quot;normal&quot;. If it's not, you will have to edit the generated script.</p>
</div>
<div class="section" id="building-optimized">
<h2><a class="toc-backref" href="#id8">Building Optimized</a></h2>
<p>There are two facets to running optimized: gathering <tt class="docutils literal">.pyo</tt>'s, and setting the
<tt class="docutils literal">Py_OptimizeFlag</tt>. Installer will gather <tt class="docutils literal">.pyo</tt>'s if it is run optimized:</p>
<pre class="literal-block">
python -O pyinstaller.py ...
</pre>
<p>The <tt class="docutils literal">Py_OptimizeFlag</tt> will be set if you use a <tt class="docutils literal"><span class="pre">('O','','OPTION')</span></tt> in one of
the <tt class="docutils literal">TOCs</tt> building the <tt class="docutils literal">EXE</tt>:</p>
<pre class="literal-block">
exe = EXE(pyz,
          a.scripts + [('O','','OPTION')],
          ...
</pre>
<p>See <a class="reference internal" href="#spec-files">Spec Files</a> for details.</p>
</div>
<div class="section" id="a-note-on-using-upx">
<h2><a class="toc-backref" href="#id9">A Note on using UPX</a></h2>
<p>On both Windows and Linux, UPX can give truly startling compression - the days
of fitting something useful on a diskette are not gone forever! Installer has
been tested with many UPX versions without problems. Just get it and install it
on your PATH.</p>
<p>For Windows, there is a problem of compatibility between UPX and executables
generated by Microsoft Visual Studio .NET 2003 (or the equivalent free
toolkit available for download). This is especially worrisome for users of
Python 2.4+, where most extensions (and Python itself) are compiled with that
compiler. This issue has been fixed in later beta versions of UPX, so you
will need at least UPX 1.92 beta. <tt class="docutils literal">pyinstaller.py</tt> will check this for you
and complain if you have an older version of UPX and you are using Python 2.4.</p>
<div class="sidebar">
<p class="first sidebar-title">UPX and Unix</p>
<p class="last">Under UNIX, old versions of UPX were not able to expand and execute the
executable in memory, and they were extracting it into a temporary file
in the filesystem, before spawning it. This is no longer valid under Linux,
but the information in this paragraph still needs to be updated.</p>
</div>
<p>For Linux, a bit more discussion is in order. First, UPX is only useful on
executables, not shared libs. Installer accounts for that, but to get the full
benefit, you might rebuild Python with more things statically linked.</p>
<p>More importantly, when <tt class="docutils literal">run</tt> finds that its <tt class="docutils literal">sys.argv[0]</tt> does not contain a path,
it will use <tt class="docutils literal">/proc/pid/exe</tt> to find itself (if it can). This happens, for
example, when executed by Apache. If it has been upx-ed, this symbolic link
points to the tempfile created by the upx stub and <cite>PyInstaller</cite> will fail (please
see the UPX docs for more information). So for now, at least, you can't use upx
for CGI's executed by Apache. Otherwise, you can ignore the warnings in the UPX
docs, since what PyInstaller opens is the executable Installer created, not the
temporary upx-created executable.</p>
</div>
<div class="section" id="accessing-data-files">
<h2><a class="toc-backref" href="#id10">Accessing Data Files</a></h2>
<p>If your application needs to access data files, e.g configuration
files or icons images, you need some minor changes to you application
and you need to collect the file into distribution directory tree (in
<cite>--onedir</cite> mode) resp. into the executable (in <cite>--onefile</cite> mode).</p>
<div class="section" id="adopt-your-application">
<h3><a class="toc-backref" href="#id11">Adopt your application</a></h3>
<p>Instead of:</p>
<pre class="literal-block">
basedir = os.path.dirname(__file__)
</pre>
<p>use:</p>
<pre class="literal-block">
if getattr(sys, 'frozen', None):
     basedir = sys._MEIPASS
else:
     basedir = os.path.dirname(__file__)
</pre>
<p><tt class="docutils literal">sys._MEIPASS</tt> points in <cite>--onedir</cite> mode to the directory containing
the created executable and in <cite>--onefile</cite> mode to the temporary directory
where binaries get extracted.</p>
</div>
<div class="section" id="collect-your-data-files">
<h3><a class="toc-backref" href="#id12">Collect your data files</a></h3>
<p>Collecting the data-files is easy: pass a list of your data files (in
<tt class="docutils literal">TOC</tt> format) to the <tt class="docutils literal">COLLECT</tt>. The <tt class="docutils literal">name</tt> in the <tt class="docutils literal">(name, path,
'DATA')</tt> tuple can be a relative path name.</p>
<p>Then, at runtime, you can use code like this to find the file:</p>
<pre class="literal-block">
os.path.join(basedir, relativename)
</pre>
<p>In a <tt class="docutils literal"><span class="pre">--onedir</span></tt> distribution, the files will listed in the
<tt class="docutils literal">COLLECT</tt> will show up in the distribution directory tree, so you
can simply pack them into your isntaller or distribution archive.</p>
<p>In a <tt class="docutils literal"><span class="pre">--onefile</span></tt> distribution, data files are bundled within the
executable and then at runtime extracted into the work directory. This
is done by the C code which is also able to reconstruct directory
trees.</p>
</div>
</div>
<div class="section" id="how-one-file-mode-works">
<h2><a class="toc-backref" href="#id13">How one-file mode works</a></h2>
<div class="sidebar">
<p class="first sidebar-title">Bootloader</p>
<p class="last">The bootloader (also known as <em>stub</em> in literature) is the small program
which starts up your packaged program. Usually, the archive containing the
bytecoded modules of your program is simply appended to it. See
<a class="reference internal" href="#self-extracting-executables">Self-extracting executables</a> for more details on the process.</p>
</div>
<p>A <tt class="docutils literal"><span class="pre">--onefile</span></tt> works by packing all the shared libs / dlls into the archive
attached to the bootloader executable (or next to the executable in a non-elf
configuration). When first started, it finds that it needs to extract these
files before it can run &quot;for real&quot;. That's because locating and loading a
shared lib or linked-in dll is a system level action, not user-level. With
PyInstaller 2.0 it always uses a temporary directory (<tt class="docutils literal">_MEIXXXXX</tt>,
where <tt class="docutils literal">XXXXX</tt> is a random number to avoid conflicts) in the
user's temp directory. It then executes itself again, setting things up so
the system will be able to load the shared libs / dlls. When execution is
complete, it recursively removes the entire directory it created.</p>
<p>The temporary directory is exported to the program's environment as
<tt class="docutils literal">sys._MEIPASS</tt>. This can be used in case you manually modified
the spec file to tell PyInstaller to add additional files (eg: data files)
within the executable (see also <a class="reference internal" href="#accessing-data-files">Accessing Data Files</a>).</p>
<p>This has a number of implications:</p>
<ul class="simple">
<li>You can run multiple copies - they won't collide.</li>
<li>Running multiple copies will be rather expensive to the system (nothing is
shared).</li>
<li>On Windows, using Task Manager to kill the parent process will leave the
directory behind.</li>
<li>On *nix, a kill -9 (or crash) will leave the directory behind.</li>
<li>Otherwise, on both platforms, the directory will be recursively deleted.</li>
<li>So any files you might create in <tt class="docutils literal">sys._MEIPASS</tt> will be deleted.</li>
<li>The executable can be in a protected or read-only directory.</li>
</ul>
<p><strong>Notes for *nix users</strong>: Take notice that if the executable does a setuid root,
a determined hacker could possibly (given enough tries) introduce a malicious
lookalike of one of the shared libraries during the hole between when the
library is extracted into the temporary directory and when it gets loaded
by the execvp'd process. So maybe you shouldn't do setuid root programs
using <tt class="docutils literal"><span class="pre">--onefile</span></tt>. <strong>In fact, we do not recomend the use of --onefile
on setuid programs.</strong></p>
</div>
<div class="section" id="egg-files-and-setuptools">
<h2><a class="toc-backref" href="#id14">.egg files and setuptools</a></h2>
<p><a class="reference external" href="http://peak.telecommunity.com/DevCenter/setuptools">setuptools</a> is a distutils extensions which provide many benefits, including
the ability to distribute the extension as <tt class="docutils literal">eggs</tt>. Together with the
nifty <a class="reference external" href="http://peak.telecommunity.com/DevCenter/EasyInstall">easy_install</a> (a tool which automatically locates, downloads and
installs Python extensions), <tt class="docutils literal">eggs</tt> are becoming more and more
widespread as a way for distributing Python extensions.</p>
<p><tt class="docutils literal">eggs</tt> can be either files or directories. An <tt class="docutils literal">egg</tt> directory is basically
a standard Python package, with some additional metadata that can be used for
advanced <a class="reference external" href="http://peak.telecommunity.com/DevCenter/setuptools">setuptools</a> features like entry-points. An <tt class="docutils literal">egg</tt> file is simply a
ZIP file, and it works as a package as well because Python 2.3+ is able to
transparently import modules stored within ZIP files.</p>
<p><cite>PyInstaller</cite> supports <tt class="docutils literal">eggs</tt> at a good level. In fact:</p>
<ul class="simple">
<li>It is able to follow dependencies within <tt class="docutils literal">eggs</tt> (both files and directories).
So if your program imports a package shipped in <tt class="docutils literal">egg</tt> format, and this package
requires additional libraries, <cite>PyInstaller</cite> will correctly include everything
within the generated executable.</li>
<li><tt class="docutils literal"><span class="pre">egg-files</span></tt> are fully supported. To let everything works (entry-points,
<tt class="docutils literal">pkg_resource</tt> library, etc.), <cite>PyInstaller</cite> either copy the <tt class="docutils literal"><span class="pre">egg-files</span></tt>
into the distribution directory (in one-dir mode) or packs them as-is within
the generated executable and unpack them at startup into the temporary directory
(see <a class="reference internal" href="#how-one-file-mode-works">How one-file mode works</a>).</li>
<li><tt class="docutils literal"><span class="pre">egg-directories</span></tt> are partially supported. In fact, <cite>PyInstaller</cite> at build
time treat them as regular package. This means that all advanced features requiring
<tt class="docutils literal">egg</tt> metadatas will not work.</li>
</ul>
<p>Improved support for <tt class="docutils literal">eggs</tt> is planned for a future release of <cite>PyInstaller</cite>.</p>
</div>
<div class="section" id="multipackage-function">
<h2><a class="toc-backref" href="#id15">Multipackage function</a></h2>
<p>Some applications are made of several different binaries, that might rely on the same
third-party libraries and/or share lots of code. When packaging such applications, it
would be a pity to treat each application binary separately and repackage all its
dependencies, potentially duplicating lots of code and libraries.</p>
<p>With Pyinstaller, you can use the multipackage feature to create multiple binaries that
might share libraries among themselves: each dependency is packaged only once in one of
the binaries, while the others simply have an &quot;external reference&quot; to it, that tells them
to go finding that dependency in the binary contains it.</p>
<p>The easiest way to access this function is to simply pass multiple script files to
<tt class="docutils literal">pyinstaller.py</tt> (or <tt class="docutils literal">utils/Makespec.py`</tt>). It will generate a spec file that contains
a call to the <a class="reference internal" href="#merge">MERGE</a> function to basically merge dependencies across the different scripts.</p>
<p>The order of the scripts on the command line (and within the <a class="reference internal" href="#merge">MERGE</a>
function) matters: given each library, PyInstaller will package common dependencies on the
leftmost script that first needs that dependency. You might want to tweak the order of
the script files accordingly.</p>
<p>Notice that the external references between binaries are hard-coded with respect to the
paths on the disk in which they are created in the output directory, and cannot be rearranged:
thus, if you use a one-file deploy, you will need to place all binaries in the same directory
when you install your application. Similarly, if you use one-dir deploy, you will need to
install all the binary directories within the same parent directory.</p>
<p>There are multipackage examples in the <tt class="docutils literal">buildtests/multipackage</tt> directory.</p>
</div>
</div>
<div class="section" id="pyinstaller-utilities">
<h1><a class="toc-backref" href="#id16">PyInstaller Utilities</a></h1>
<div class="section" id="archiveviewer">
<h2><a class="toc-backref" href="#id17">ArchiveViewer</a></h2>
<pre class="literal-block">
python utils/ArchiveViewer.py &lt;archivefile&gt;
</pre>
<p>ArchiveViewer lets you examine the contents of any archive build with
<cite>PyInstaller</cite> or executable (PYZ, PKG or exe). Invoke it with the target as the
first arg (It has been set up as a Send-To so it shows on the context menu in
Explorer). The archive can be navigated using these commands:</p>
<dl class="docutils">
<dt>O &lt;nm&gt;</dt>
<dd>Open the embedded archive &lt;nm&gt; (will prompt if omitted).</dd>
<dt>U</dt>
<dd>Go up one level (go back to viewing the embedding archive).</dd>
<dt>X &lt;nm&gt;</dt>
<dd>Extract nm (will prompt if omitted). Prompts for output filename. If none
given, extracted to stdout.</dd>
<dt>Q</dt>
<dd>Quit.</dd>
</dl>
<p>Futhermore ArchiveViewer has some simple console commands:</p>
<table class="docutils option-list" frame="void" rules="none">
<col class="option" />
<col class="description" />
<tbody valign="top">
<tr><td class="option-group">
<kbd><span class="option">-h</span>, <span class="option">--help</span></kbd></td>
<td>Show help.</td></tr>
<tr><td class="option-group">
<kbd><span class="option">-l</span>, <span class="option">--log</span></kbd></td>
<td>Quick contents log.</td></tr>
<tr><td class="option-group">
<kbd><span class="option">-b</span>, <span class="option">--brief</span></kbd></td>
<td>Print a python evaluable list of contents filenames.</td></tr>
<tr><td class="option-group" colspan="2">
<kbd><span class="option">-r</span>, <span class="option">--recursive</span></kbd></td>
</tr>
<tr><td>&nbsp;</td><td>Used with -l or -b, applies recusive behaviour.</td></tr>
</tbody>
</table>
</div>
<div class="section" id="bindepend">
<h2><a class="toc-backref" href="#id18">BinDepend</a></h2>
<pre class="literal-block">
python utils/BinDepend.py &lt;executable_or_dynamic_library&gt;
</pre>
<p>BinDepend will analyze the executable you pass to it, and write to stdout all
its binary dependencies. This is handy to find out which DLLs are required by
an executable or another DLL. This module is used by <cite>PyInstaller</cite> itself to
follow the chain of dependencies of binary extensions and make sure that all
of them get included in the final package.</p>
</div>
<div class="section" id="grabversion-windows">
<h2><a class="toc-backref" href="#id19">GrabVersion (Windows)</a></h2>
<pre class="literal-block">
python utils/GrabVersion.py &lt;executable_with_version_resource&gt;
</pre>
<p>GrabVersion outputs text which can be eval'ed by <tt class="docutils literal">versionInfo.py</tt> to reproduce
a version resource. Invoke it with the full path name of a Windows executable
(with a version resource) as the first argument. If you cut &amp; paste (or
redirect to a file), you can then edit the version information. The edited
text file can be used in a <tt class="docutils literal">version = myversion.txt</tt> option on any executable
in an <cite>PyInstaller</cite> spec file.</p>
<p>This was done in this way because version resources are rather strange beasts,
and fully understanding them is probably impossible. Some elements are
optional, others required, but you could spend unbounded amounts of time
figuring this out, because it's not well documented. When you view the version
tab on a properties dialog, there's no straightforward relationship between
how the data is displayed and the structure of the resource itself. So the
easiest thing to do is find an executable that displays the kind of
information you want, grab it's resource and edit it. Certainly easier than
the Version resource wizard in VC++.</p>
</div>
<div class="section" id="analyzing-dependencies">
<h2><a class="toc-backref" href="#id20">Analyzing Dependencies</a></h2>
<p>You can interactively track down dependencies, including getting
cross-references by using <tt class="docutils literal">mf.py</tt>, documented in section <a class="reference internal" href="#mf-py-a-modulefinder-replacement">mf.py: A modulefinder
Replacement</a></p>
</div>
</div>
<div class="section" id="spec-files">
<h1><a class="toc-backref" href="#id21">Spec Files</a></h1>
<div class="section" id="introduction">
<h2><a class="toc-backref" href="#id22">Introduction</a></h2>
<p>When you run <tt class="docutils literal">utils/Makespec.py</tt> (documented
in section <a class="reference internal" href="#a-spec-file-for-your-project">A spec file for your project</a>), it generates a
spec file for you. In fact,
you can think of <tt class="docutils literal">utils/Makespec.py</tt> just like a wizard that lets you generate
a standard spec file for most standard usages. But advanced users can
learn to edit spec files to fully customize PyInstaller behaviour to
their needs, giving beyond the standard settings provided by the wizard.</p>
<p>Spec files are in Python syntax. They are evaluated by pyinstaller.py. A simplistic
spec file might look like this:</p>
<pre class="literal-block">
a = Analysis(['myscript.py'])
pyz = PYZ(a.pure)
exe = EXE(pyz, a.scripts, a.binaries, name=&quot;myapp.exe&quot;)
</pre>
<p>This creates a single file deployment with all binaries (extension modules and
their dependencies) packed into the executable.</p>
<p>A simplistic single directory deployment might look like this:</p>
<pre class="literal-block">
a = Analysis(['myscript.py'])
pyz = PYZ(a.pure)
exe = EXE(a.scripts, pyz, name=&quot;myapp.exe&quot;, exclude_binaries=1)
dist = COLLECT(exe, a.binaries, name=&quot;dist&quot;)
</pre>
<p>Note that neither of these examples are realistic. If you want to
start hacking a spec file, use <tt class="docutils literal">utils/Makespec.py</tt> to create a basic specfile,
and tweak it (if necessary) from there.</p>
<p>All of the classes you see above are subclasses of <tt class="docutils literal">Build.Target</tt>. A Target acts
like a rule in a makefile. It knows enough to cache its last inputs and
outputs. If its inputs haven't changed, it can assume its outputs wouldn't
change on recomputation. So a spec file acts much like a makefile, only
rebuilding as much as needs rebuilding. This means, for example, that if you
change an <tt class="docutils literal">EXE</tt> from <tt class="docutils literal">debug=1</tt> to <tt class="docutils literal">debug=0</tt>, the rebuild will be nearly
instantaneous.</p>
<p>The high level view is that an <tt class="docutils literal">Analysis</tt> takes a list of scripts as input,
and generates three &quot;outputs&quot;, held in attributes named <tt class="docutils literal">scripts</tt>, <tt class="docutils literal">pure</tt>
and <tt class="docutils literal">binaries</tt>. A <tt class="docutils literal">PYZ</tt> (a <tt class="docutils literal">.pyz</tt> archive) is built from the modules in
pure. The <tt class="docutils literal">EXE</tt> is built from the <tt class="docutils literal">PYZ</tt>, the scripts and, in the case of a
single-file deployment, the binaries. In a single-directory deployment, a
directory is built containing a slim executable and the binaries.</p>
</div>
<div class="section" id="toc-class-table-of-contents">
<h2><a class="toc-backref" href="#id23">TOC Class (Table of Contents)</a></h2>
<p>Before you can do much with a spec file, you need to understand the
<tt class="docutils literal">TOC</tt> (Table Of Contents) class.</p>
<p>A <tt class="docutils literal">TOC</tt> appears to be a list of tuples of the form (name, path, typecode).
In fact, it's an ordered set, not a list. A TOC contains no duplicates, where
uniqueness is based on name only. Furthermore, within this constraint, a TOC
preserves order.</p>
<p>Besides the normal list methods and operations, TOC supports taking differences
and intersections (and note that adding or extending is really equivalent to
union). Furthermore, the operations can take a real list of tuples on the right
hand side. This makes excluding modules quite easy. For a pure Python module:</p>
<pre class="literal-block">
pyz = PYZ(a.pure - [('badmodule', '', '')])
</pre>
<p>or for an extension module in a single-directory deployment:</p>
<pre class="literal-block">
dist = COLLECT(..., a.binaries - [('badmodule', '', '')], ...)
</pre>
<p>or for a single-file deployment:</p>
<pre class="literal-block">
exe = EXE(..., a.binaries - [('badmodule', '', '')], ...)
</pre>
<p>To add files to a TOC, you need to know about the typecodes (or the step using
the TOC won't know what to do with the entry).</p>
<table border="1" class="docutils">
<colgroup>
<col width="12%" />
<col width="44%" />
<col width="19%" />
<col width="25%" />
</colgroup>
<thead valign="bottom">
<tr><th class="head"><strong>typecode</strong></th>
<th class="head"><strong>description</strong></th>
<th class="head"><strong>name</strong></th>
<th class="head"><strong>path</strong></th>
</tr>
</thead>
<tbody valign="top">
<tr><td>'EXTENSION'</td>
<td>An extension module.</td>
<td>Python internal name.</td>
<td>Full path name in build.</td>
</tr>
<tr><td>'PYSOURCE'</td>
<td>A script.</td>
<td>Python internal name.</td>
<td>Full path name in build.</td>
</tr>
<tr><td>'PYMODULE'</td>
<td>A pure Python module (including __init__ modules).</td>
<td>Python internal name.</td>
<td>Full path name in build.</td>
</tr>
<tr><td>'PYZ'</td>
<td>A .pyz archive (archive_rt.ZlibArchive).</td>
<td>Runtime name.</td>
<td>Full path name in build.</td>
</tr>
<tr><td>'PKG'</td>
<td>A pkg archive (carchive4.CArchive).</td>
<td>Runtime name.</td>
<td>Full path name in build.</td>
</tr>
<tr><td>'BINARY'</td>
<td>A shared library.</td>
<td>Runtime name.</td>
<td>Full path name in build.</td>
</tr>
<tr><td>'DATA'</td>
<td>Aribitrary files.</td>
<td>Runtime name.</td>
<td>Full path name in build.</td>
</tr>
<tr><td>'OPTION'</td>
<td>A runtime runtime option (frozen into the executable).</td>
<td>The option.</td>
<td>Unused.</td>
</tr>
</tbody>
</table>
<p>You can force the include of any file in much the same way you do excludes:</p>
<pre class="literal-block">
collect = COLLECT(a.binaries +
          [('readme', '/my/project/readme', 'DATA')], ...)
</pre>
<p>or even:</p>
<pre class="literal-block">
collect = COLLECT(a.binaries,
          [('readme', '/my/project/readme', 'DATA')], ...)
</pre>
<p>(that is, you can use a list of tuples in place of a <tt class="docutils literal">TOC</tt> in most cases).</p>
<p>There's not much reason to use this technique for <tt class="docutils literal">PYSOURCE</tt>, since an <tt class="docutils literal">Analysis</tt>
takes a list of scripts as input. For <tt class="docutils literal">PYMODULEs</tt> and <tt class="docutils literal">EXTENSIONs</tt>, the hook
mechanism discussed here is better because you won't have to remember how you
got it working next time.</p>
<p>This technique is most useful for data files (see the <tt class="docutils literal">Tree</tt> class below for a
way to build a <tt class="docutils literal">TOC</tt> from a directory tree), and for runtime options. The options
the run executables understand are:</p>
<table border="1" class="docutils">
<colgroup>
<col width="9%" />
<col width="13%" />
<col width="18%" />
<col width="60%" />
</colgroup>
<thead valign="bottom">
<tr><th class="head"><strong>Option</strong></th>
<th class="head"><strong>Description</strong></th>
<th class="head"><strong>Example</strong></th>
<th class="head"><strong>Notes</strong></th>
</tr>
</thead>
<tbody valign="top">
<tr><td>v</td>
<td>Verbose imports</td>
<td>('v', '', 'OPTION')</td>
<td>Same as Python -v ...</td>
</tr>
<tr><td>u</td>
<td>Unbuffered stdio</td>
<td>('u', '', 'OPTION')</td>
<td>Same as Python -u ...</td>
</tr>
<tr><td>W spec</td>
<td>Warning option</td>
<td>('W ignore', '', 'OPTION')</td>
<td>Python 2.1+ only.</td>
</tr>
<tr><td>s</td>
<td>Use site.py</td>
<td>('s', '', 'OPTION')</td>
<td>The opposite of Python's -S flag. Note that site.py must be in the executable's directory to be used.</td>
</tr>
</tbody>
</table>
<p>Advanced users should note that by using set differences and intersections, it
becomes possible to factor out common modules, and deploy a project containing
multiple executables with minimal redundancy. You'll need some top level code
in each executable to mount the common <tt class="docutils literal">PYZ</tt>.</p>
</div>
<div class="section" id="target-subclasses">
<h2><a class="toc-backref" href="#id24">Target Subclasses</a></h2>
<div class="section" id="analysis">
<h3><a class="toc-backref" href="#id25">Analysis</a></h3>
<pre class="literal-block">
Analysis(scripts, pathex=None, hookspath=None, excludes=None)
</pre>
<dl class="docutils">
<dt><tt class="docutils literal">scripts</tt></dt>
<dd>a list of scripts specified as file names.</dd>
<dt><tt class="docutils literal">pathex</tt></dt>
<dd>an optional list of paths to be searched before sys.path.</dd>
<dt><tt class="docutils literal">hiddenimports</tt></dt>
<dd>an optional list of additional (hidden) modules to include. Please
refer to <a class="reference internal" href="#listing-hidden-imports">Listing Hidden Imports</a> for details.</dd>
<dt><tt class="docutils literal">hookspath</tt></dt>
<dd>an optional list of additional paths to search for hooks
(hook-modules). Please refer to <a class="reference internal" href="#listing-hidden-imports">Listing Hidden Imports</a> for details.</dd>
<dt><tt class="docutils literal">excludes</tt></dt>
<dd>an optional list of module or package names (their Python names, not path
names) that will be ignored (as though they were not found).</dd>
</dl>
<p>An Analysis has five outputs, all <tt class="docutils literal">TOCs</tt> accessed as attributes of the <tt class="docutils literal">Analysis</tt>.</p>
<dl class="docutils">
<dt><tt class="docutils literal">scripts</tt></dt>
<dd>The scripts you gave Analysis as input, with any runtime hook scripts
prepended.</dd>
<dt><tt class="docutils literal">pure</tt></dt>
<dd>The pure Python modules.</dd>
<dt><tt class="docutils literal">binaries</tt></dt>
<dd>The extension modules and their dependencies. The secondary dependencies are
filtered. On Windows, a long list of MS dlls are excluded. On Linux/Unix,
any shared lib in <tt class="docutils literal">/lib</tt> or <tt class="docutils literal">/usr/lib</tt> is excluded.</dd>
<dt><tt class="docutils literal">datas</tt></dt>
<dd>Data-file dependencies. These are data-file that are found to be needed by
modules. They can be anything: plugins, font files, etc.</dd>
<dt><tt class="docutils literal">zipfiles</tt></dt>
<dd>The zipfiles dependencies (usually <tt class="docutils literal"><span class="pre">egg-files</span></tt>).</dd>
</dl>
</div>
<div class="section" id="pyz">
<h3><a class="toc-backref" href="#id26">PYZ</a></h3>
<pre class="literal-block">
PYZ(toc, name=None, level=9)
</pre>
<dl class="docutils">
<dt><tt class="docutils literal">toc</tt></dt>
<dd>a <tt class="docutils literal">TOC</tt>, normally an <tt class="docutils literal">Analysis.pure</tt>.</dd>
<dt><tt class="docutils literal">name</tt></dt>
<dd>A filename for the <tt class="docutils literal">.pyz</tt>. Normally not needed, as the generated name will do fine.</dd>
<dt><tt class="docutils literal">level</tt></dt>
<dd>The Zlib compression level to use. If 0, the zlib module is not required.</dd>
</dl>
</div>
<div class="section" id="pkg">
<h3><a class="toc-backref" href="#id27">PKG</a></h3>
<p>Generally, you will not need to create your own <tt class="docutils literal">PKGs</tt>, as the <tt class="docutils literal">EXE</tt> will do it for
you. This is one way to include read-only data in a single-file deployment,
however.</p>
<pre class="literal-block">
PKG(toc, name=None, cdict=None, exclude_binaries=0)
</pre>
<dl class="docutils">
<dt><tt class="docutils literal">toc</tt></dt>
<dd>a <tt class="docutils literal">TOC</tt>.</dd>
<dt><tt class="docutils literal">name</tt></dt>
<dd>a filename for the <tt class="docutils literal">PKG</tt> (optional).</dd>
<dt><tt class="docutils literal">cdict</tt></dt>
<dd>a dictionary that specifies compression by typecode. For example, <tt class="docutils literal">PYZ</tt> is
left uncompressed so that it can be accessed inside the <tt class="docutils literal">PKG</tt>. The default
uses sensible values. If zlib is not available, no compression is used.</dd>
<dt><tt class="docutils literal">exclude_binaries</tt></dt>
<dd>If 1, <tt class="docutils literal">EXTENSIONs</tt> and <tt class="docutils literal">BINARYs</tt> will be left out of the <tt class="docutils literal">PKG</tt>, and
forwarded to its container (usually a <tt class="docutils literal">COLLECT</tt>).</dd>
</dl>
</div>
<div class="section" id="exe">
<h3><a class="toc-backref" href="#id28">EXE</a></h3>
<pre class="literal-block">
EXE(*args, **kws)
</pre>
<dl class="docutils">
<dt><tt class="docutils literal">args</tt></dt>
<dd>One or more arguments which are either <tt class="docutils literal">TOCs</tt> or <tt class="docutils literal">Targets</tt>.</dd>
<dt><tt class="docutils literal">kws</tt></dt>
<dd><p class="first">Possible keyword arguments:</p>
<dl class="last docutils">
<dt><tt class="docutils literal">console</tt></dt>
<dd>Always 1 on Linux/unix. On Windows, governs whether to use the console
executable, or the Windows subsystem executable.</dd>
<dt><tt class="docutils literal">debug</tt></dt>
<dd>Setting to 1 gives you progress messages from the executable (for a
<tt class="docutils literal">console=0</tt>, these will be annoying MessageBoxes).</dd>
<dt><tt class="docutils literal">name</tt></dt>
<dd>The filename for the executable.</dd>
<dt><tt class="docutils literal">exclude_binaries</tt></dt>
<dd>Forwarded to the <tt class="docutils literal">PKG</tt> the <tt class="docutils literal">EXE</tt> builds.</dd>
<dt><tt class="docutils literal">icon</tt></dt>
<dd>Windows NT family only. <tt class="docutils literal"><span class="pre">icon='myicon.ico'</span></tt> to use an icon file, or
<tt class="docutils literal"><span class="pre">icon='notepad.exe,0'</span></tt> to grab an icon resource.</dd>
<dt><tt class="docutils literal">version</tt></dt>
<dd>Windows NT family only. <tt class="docutils literal"><span class="pre">version='myversion.txt'</span></tt>. Use <tt class="docutils literal">GrabVersion.py</tt> to
steal a version resource from an executable, and then edit the ouput to
create your own. (The syntax of version resources is so arcane that I
wouldn't attempt to write one from scratch.)</dd>
<dt><tt class="docutils literal">append_pkg</tt></dt>
<dd>If <tt class="docutils literal">True</tt>, then append the PKG archive to the EXE. If <tt class="docutils literal">False</tt>,
place the PKG archive in a separate file <tt class="docutils literal">exename.pkg</tt>.
The default depends
on whether Make.py was given the <tt class="docutils literal"><span class="pre">-n</span></tt> argument
when building the loader. The default is <tt class="docutils literal">True</tt> on Windows.
On non-ELF platforms where concatenating arbitrary data to
an executable does not work, <tt class="docutils literal">append_pkg</tt> must be set to <tt class="docutils literal">False</tt>.</dd>
</dl>
</dd>
</dl>
</div>
<div class="section" id="dll">
<h3><a class="toc-backref" href="#id29">DLL</a></h3>
<p>On Windows, this provides support for doing in-process COM servers. It is not
generalized. However, embedders can follow the same model to build a special
purpose DLL so the Python support in their app is hidden. You will need to
write your own dll, but thanks to Allan Green for refactoring the C code and
making that a managable task.</p>
</div>
<div class="section" id="collect">
<h3><a class="toc-backref" href="#id30">COLLECT</a></h3>
<pre class="literal-block">
COLLECT(*args, **kws)
</pre>
<dl class="docutils">
<dt><tt class="docutils literal">args</tt></dt>
<dd>One or more arguments which are either <tt class="docutils literal">TOCs</tt> or <tt class="docutils literal">Targets</tt>.</dd>
<dt><tt class="docutils literal">kws</tt></dt>
<dd><p class="first">Possible keyword arguments:</p>
<dl class="last docutils">
<dt><tt class="docutils literal">name</tt></dt>
<dd>The name of the directory to be built.</dd>
</dl>
</dd>
</dl>
</div>
<div class="section" id="tree">
<h3><a class="toc-backref" href="#id31">Tree</a></h3>
<pre class="literal-block">
Tree(root, prefix=None, excludes=None)
</pre>
<dl class="docutils">
<dt><tt class="docutils literal">root</tt></dt>
<dd>The root of the tree (on the build system).</dd>
<dt><tt class="docutils literal">prefix</tt></dt>
<dd>Optional prefix to the names on the target system.</dd>
<dt><tt class="docutils literal">excludes</tt></dt>
<dd><p class="first">A list of names to exclude. Two forms are allowed:</p>
<dl class="last docutils">
<dt><tt class="docutils literal">name</tt></dt>
<dd>files with this basename will be excluded (do not include the path).</dd>
<dt><tt class="docutils literal">*.ext</tt></dt>
<dd>any file with the given extension will be excluded.</dd>
</dl>
</dd>
</dl>
<p>Since a <tt class="docutils literal">Tree</tt> is a <tt class="docutils literal">TOC</tt>, you can also use the exclude technique described above
in the section on <tt class="docutils literal">TOCs</tt>.</p>
</div>
<div class="section" id="merge">
<h3><a class="toc-backref" href="#id32">MERGE</a></h3>
<p>With the MERGE function we can create a group of interdependent packages.</p>
<pre class="literal-block">
MERGE(*args)
</pre>
<dl class="docutils">
<dt><tt class="docutils literal">*args</tt></dt>
<dd>This is a list of tuples. The first element of the tuple is an analysis object,
the second one is the script name without extension and the third one is the final name.</dd>
</dl>
<p>The <tt class="docutils literal">MERGE</tt> function filters the analysis to avoid duplication of libraries and modules.
As a result the packages generated will be connected. Furthermore, to ensure the consistency
of dependencies, it replaces the temporary names with the actual names.
MERGE is used after the analysis phase and before <tt class="docutils literal">EXE</tt> and <tt class="docutils literal">COLLECT</tt>.</p>
<p>Here is spec file example with <tt class="docutils literal">MERGE</tt> function:</p>
<pre class="literal-block">
## Where the package folders will be built, and the shortcuts will reside
TargetDir = os.path.abspath(os.path.join('..','..','Client','Programs'))

## The application names
AppNames = [d for d in os.listdir(os.getcwd())
            if os.path.isdir(d)
            and d[0]!='.'
            and d[0:6]!='Common'
            and d != 'build'
            and d != 'dummy']

## Build MERGE arguments (analysis object, script base name, final exe path)
#  Start with the dummy package
Analyses = [(Analysis([os.path.join('dummy','dummy.py')]),
             'dummy', os.path.join('dummy','dummy.exe'))
            ]

#  NOTE: this assumes that the main script in each is appname.pyw in the appname folder
Analyses += [(Analysis([os.path.join(appname, appname + '.pyw')]),
              appname, os.path.join(appname,appname+'.exe'))
             for appname in AppNames]

## Merge all the dependencies
MERGE(*Analyses)

## Build each app
for anal, basename, exename in Analyses:
    pyz = PYZ(anal.pure)
    exe = EXE(pyz,
              anal.scripts,
              anal.dependencies,
              exclude_binaries=1,
              name=exename,
              version='FalconVersion.txt',
              debug=False,
              strip=False,
              upx=True,
              console=False )
    dist = COLLECT(exe,
                   anal.binaries,
                   anal.zipfiles,
                   anal.datas,
                   strip=False,
                   ###upx=True if (basename == 'dummy') else False,
                   upx=False,
                   name=os.path.join(TargetDir,basename))
</pre>
</div>
</div>
</div>
<div class="section" id="when-things-go-wrong">
<h1><a class="toc-backref" href="#id33">When Things Go Wrong</a></h1>
<div class="section" id="recipes-and-instructions-for-special-modules">
<h2><a class="toc-backref" href="#id34">Recipes and Instructions for special Modules</a></h2>
<p>Code examples for some modules needing special care and some common
issues are available on our <a class="reference external" href="http://www.pyinstaller.org/wiki/Recipe">Recipe</a> web-page.</p>
</div>
<div class="section" id="finding-out-what-went-wrong">
<h2><a class="toc-backref" href="#id35">Finding out What Went Wrong</a></h2>
<div class="section" id="buildtime-warnings">
<h3><a class="toc-backref" href="#id36">Buildtime Warnings</a></h3>
<p>When an <tt class="docutils literal">Analysis</tt> step runs, it produces a warnings file (named <tt class="docutils literal">warnproject.txt</tt>)
in the spec file's directory. Generally, most of these warnings are harmless.
For example, <tt class="docutils literal">os.py</tt> (which is cross-platform) works by figuring out what
platform it is on, then importing (and rebinding names from) the appropriate
platform-specific module. So analyzing <tt class="docutils literal">os.py</tt> will produce a set of warnings
like:</p>
<pre class="literal-block">
WARNING: no module named dos (conditional import by os)
WARNING: no module named ce (conditional import by os)
WARNING: no module named os2 (conditional import by os)
</pre>
<p>Note that the analysis has detected that the import is within a conditional
block (an if statement). The analysis also detects if an import within a
function or class, (delayed) or at the top level. A top-level, non-conditional
import failure is really a hard error. There's at least a reasonable chance
that conditional and / or delayed import will be handled gracefully at runtime.</p>
<p>Ignorable warnings may also be produced when a class or function is declared in
a package (an <tt class="docutils literal">__init__.py</tt> module), and the import specifies
<tt class="docutils literal">package.name</tt>. In this case, the analysis can't tell if name is supposed to
refer to a submodule of package.</p>
<p>Warnings are also produced when an <tt class="docutils literal">__import__</tt>, <tt class="docutils literal">exec</tt> or <tt class="docutils literal">eval</tt> statement is
encountered. The <tt class="docutils literal">__import__</tt> warnings should almost certainly be investigated.
Both <tt class="docutils literal">exec</tt> and <tt class="docutils literal">eval</tt> can be used to implement import hacks, but usually their use
is more benign.</p>
<p>Any problem detected here can be handled by hooking the analysis of the module.
See <a class="reference internal" href="#listing-hidden-imports">Listing Hidden Imports</a> below for how to do it.</p>
</div>
<div class="section" id="getting-debug-messages">
<h3><a class="toc-backref" href="#id37">Getting Debug Messages</a></h3>
<p>Debug messages for PyInstaller  can be enabled by passing the <tt class="docutils literal"><span class="pre">--log-level</span></tt>
flag to the <tt class="docutils literal">pyinstaller.py</tt> script:</p>
<pre class="literal-block">
pyinstaller.py --log-level=DEBUG &lt;scriptname&gt;
</pre>
<p>Setting <tt class="docutils literal">debug=1</tt> on an <tt class="docutils literal">EXE</tt> will cause the executable to put out progress
messages (for console apps, these go to stdout; for Windows apps, these show as
MessageBoxes). This can be useful if you are doing complex packaging, or your
app doesn't seem to be starting, or just to learn how the runtime works.</p>
</div>
<div class="section" id="getting-python-s-verbose-imports">
<h3><a class="toc-backref" href="#id38">Getting Python's Verbose Imports</a></h3>
<p>You can also pass a <tt class="docutils literal"><span class="pre">-v</span></tt> (verbose imports) flag to the embedded Python. This can
be extremely useful. I usually try it even on apparently working apps, just to
make sure that I'm always getting my copies of the modules and no import has
leaked out to the installed Python.</p>
<p>You set this (like the other runtime options) by feeding a phone <tt class="docutils literal">TOC</tt> entry to
the <tt class="docutils literal">EXE</tt>. The easiest way to do this is to change the <tt class="docutils literal">EXE</tt> from:</p>
<pre class="literal-block">
EXE(..., anal.scripts, ....)
</pre>
<p>to:</p>
<pre class="literal-block">
EXE(..., anal.scripts + [('v', '', 'OPTION')], ...)
</pre>
<p>These messages will always go to <tt class="docutils literal">stdout</tt>, so you won't see them on Windows if
<tt class="docutils literal">console=0</tt>.</p>
</div>
</div>
<div class="section" id="helping-pyinstaller-find-modules">
<h2><a class="toc-backref" href="#id39">Helping PyInstaller Find Modules</a></h2>
<div class="section" id="extending-the-path">
<h3><a class="toc-backref" href="#id40">Extending the Path</a></h3>
<p>When the analysis phase cannot find needed modules, it may be that the code is
manipulating <tt class="docutils literal">sys.path</tt>. The easiest thing to do in this case is tell <tt class="docutils literal">Analysis</tt>
about the new directory through the second arg to the constructor:</p>
<pre class="literal-block">
anal = Analysis(['somedir/myscript.py'],
                ['path/to/thisdir', 'path/to/thatdir'])
</pre>
<p>In this case, the <tt class="docutils literal">Analysis</tt> will have a search path:</p>
<pre class="literal-block">
['somedir', 'path/to/thisdir', 'path/to/thatdir'] + sys.path
</pre>
<p>You can do the same when running <tt class="docutils literal">utils/Makespec.py</tt> or <tt class="docutils literal">pyinstaller.py</tt>:</p>
<pre class="literal-block">
utils/Makespec.py --paths=path/to/thisdir;path/to/thatdir ...
pyinstaller.py --paths=path/to/thisdir;path/to/thatdir ...
</pre>
<p>(on *nix, use <tt class="docutils literal">:</tt> as the path separator).</p>
</div>
<div class="section" id="listing-hidden-imports">
<h3><a class="toc-backref" href="#id41">Listing Hidden Imports</a></h3>
<p>Hidden imports are fairly common. These can occur when the code is using
<tt class="docutils literal">__import__</tt> (or, perhaps <tt class="docutils literal">exec</tt> or <tt class="docutils literal">eval</tt>), in which case you will see a warning in
the <tt class="docutils literal">warnproject.txt</tt> file. They can also occur when an extension module uses the
Python/C API to do an import, in which case Analysis can't detect
anything.</p>
<p>You
can verify that hidden import is the problem by using Python's verbose imports
flag. If the import messages say &quot;module not found&quot;, but the <tt class="docutils literal">warnproject.txt</tt>
file has no &quot;no module named...&quot; message for the same module, then the problem
is a hidden import.</p>
<div class="sidebar">
<p class="first sidebar-title">Standard hidden imports are already included!</p>
<p class="last">If you are getting worried while reading this paragraph, do not worry:
having hidden imports is the exception, not the norm! And anyway,
PyInstaller already ships with a large set of hooks that take care of
hidden imports for the most common packages out there. For instance,
<a class="reference external" href="http://www.pythonware.com/products/pil/">PIL</a>, <a class="reference external" href="http://sourceforge.net/projects/pywin32/files/">PyWin32</a>, <a class="reference external" href="http://www.riverbankcomputing.co.uk/pyqt/index.php">PyQt</a> are already taken care of.</p>
</div>
<p>Hidden imports are handled by hooking the module (the one doing the
hidden imports) at <tt class="docutils literal">Analysis</tt> time. Do this as follows:</p>
<blockquote>
<ol class="arabic">
<li><p class="first">Create a file named <tt class="docutils literal"><span class="pre">hook-module.py</span></tt> (where <cite>module</cite> is the
fully-qualified Python name, eg, <tt class="docutils literal"><span class="pre">hook-xml.dom.py</span></tt>) and place it
somewhere. Remember the place as <cite>your private hooks directory</cite>.</p>
</li>
<li><p class="first">In the .spec file, pass <cite>your private hooks directory</cite> as
<tt class="docutils literal">hookspath</tt> argument to <tt class="docutils literal">Analysis</tt> so will be searched.
Example:</p>
<pre class="literal-block">
a = Analysis(['myscript.py'], hookspath='/my/priv/hooks')
</pre>
</li>
</ol>
</blockquote>
<p>In most cases the hook module will have only one line:</p>
<pre class="literal-block">
hiddenimports = ['module1', 'module2']
</pre>
<p>When the <tt class="docutils literal">Analysis</tt> finds this file, it will proceed exactly as
though the module explicitly imported <tt class="docutils literal">module1</tt> and <tt class="docutils literal">module2</tt>.</p>
<p>If you successfully hook a publicly distributed module in this way,
please send us the hook so we can make it available to others.</p>
<p>You may want to have a look at already existing hooks in the
<tt class="docutils literal">PyInstaller.hooks</tt> package under <cite>PyInstaller</cite>'s root directory.
For full details on the analysis-time hook mechanism is in the <a class="reference internal" href="#hooks">Hooks</a>
section.</p>
</div>
<div class="section" id="extending-a-package-s-path">
<h3><a class="toc-backref" href="#id42">Extending a Package's <tt class="docutils literal">__path__</tt></a></h3>
<p>Python allows a package to extend the search path used to find modules and
sub-packages through the <tt class="docutils literal">__path__</tt> mechanism. Normally, a package's <tt class="docutils literal">__path__</tt> has
only one entry - the directory in which the <tt class="docutils literal">__init__.py</tt> was found. But
<tt class="docutils literal">__init__.py</tt> is free to extend its <tt class="docutils literal">__path__</tt> to include other directories. For
example, the <tt class="docutils literal">win32com.shell.shell</tt> module actually resolves to
<tt class="docutils literal">win32com/win32comext/shell/shell.pyd</tt>. This is because <tt class="docutils literal">win32com/__init__.py</tt>
appends <tt class="docutils literal"><span class="pre">../win32comext</span></tt> to its <tt class="docutils literal">__path__</tt>.</p>
<p>Because the <tt class="docutils literal">__init__.py</tt> is not actually run during an analysis, we use the same
hook mechanism we use for hidden imports. A static list of names won't do,
however, because the new entry on <tt class="docutils literal">__path__</tt> may well require computation. So
<tt class="docutils literal"><span class="pre">hook-module.py</span></tt> should define a method <tt class="docutils literal">hook(mod)</tt>. The mod argument is an
instance of <tt class="docutils literal">mf.Module</tt> which has (more or less) the same attributes as a real
module object. The hook function should return a <tt class="docutils literal">mf.Module</tt> instance - perhaps
a brand new one, but more likely the same one used as an arg, but mutated.
See <a class="reference internal" href="#mf-py-a-modulefinder-replacement">mf.py: A Modulefinder Replacement</a> for details, and <a class="reference external" href="http://www.pyinstaller.org/browser/trunk/PyInstaller/hooks/hook-win32com.py?rev=latest">PyInstaller/hooks/hook-win32com.py</a>
for an example.</p>
<p>Note that manipulations of <tt class="docutils literal">__path__</tt> hooked in this way apply to the analysis,
and only the analysis. That is, at runtime <tt class="docutils literal">win32com.shell</tt> is resolved the same
way as <tt class="docutils literal">win32com.anythingelse</tt>, and <tt class="docutils literal">win32com.__path__</tt> knows nothing of <tt class="docutils literal"><span class="pre">../win32comext</span></tt>.</p>
<p>Once in awhile, that's not enough.</p>
</div>
<div class="section" id="changing-runtime-behavior">
<h3><a class="toc-backref" href="#id43">Changing Runtime Behavior</a></h3>
<p>More bizarre situations can be accomodated with runtime hooks. These are small
scripts that manipulate the environment before your main script runs,
effectively providing additional top-level code to your script.</p>
<p>At the tail end of an analysis, the module list is examined for matches in
<tt class="docutils literal">support/rthooks.dat</tt>, which is the string representation of a
Python dictionary. The key is the module name, and the value is a list
of hook-script pathnames.</p>
<p>So putting an entry:</p>
<pre class="literal-block">
'somemodule': ['path/to/somescript.py'],
</pre>
<p>into <tt class="docutils literal">support/rthooks.dat</tt> is almost the same thing as doing this:</p>
<pre class="literal-block">
anal = Analysis(['path/to/somescript.py', 'main.py'], ...
</pre>
<p>except that in using the hook, <tt class="docutils literal">path/to/somescript.py</tt> will not be analyzed,
(that's not a feature - we just haven't found a sane way fit the recursion into
my persistence scheme).</p>
<p>Hooks done in this way, while they need to be careful of what they import, are
free to do almost anything. One provided hook sets things up so that win32com
can generate modules at runtime (to disk), and the generated modules can be
found in the win32com package.</p>
</div>
<div class="section" id="adapting-to-being-frozen">
<h3><a class="toc-backref" href="#id44">Adapting to being &quot;frozen&quot;</a></h3>
<p>In most sophisticated apps, it becomes necessary to figure out (at runtime)
whether you're running &quot;live&quot; or &quot;frozen&quot;. For example, you might have a
configuration file that (running &quot;live&quot;) you locate based on a module's
<tt class="docutils literal">__file__</tt> attribute. That won't work once the code is packaged up. You'll
probably want to look for it based on <tt class="docutils literal">sys.executable</tt> instead.</p>
<p>The bootloaders set <tt class="docutils literal">sys.frozen=1</tt> (and, for in-process COM servers, the
embedding DLL sets <tt class="docutils literal"><span class="pre">sys.frozen='dll'</span></tt>).</p>
<p>For really advanced users, you can access the <tt class="docutils literal">iu.ImportManager</tt> as
<tt class="docutils literal">sys.importManager</tt>. See <a class="reference internal" href="#iu-py">iu.py</a> for how you might make use of this fact.</p>
</div>
</div>
</div>
<div class="section" id="miscellaneous">
<h1><a class="toc-backref" href="#id45">Miscellaneous</a></h1>
<div class="section" id="self-extracting-executables">
<h2><a class="toc-backref" href="#id46">Self-extracting executables</a></h2>
<p>The ELF executable format (Windows, Linux and some others) allows arbitrary
data to be concatenated to the end of the executable without disturbing its
functionality. For this reason, a <tt class="docutils literal">CArchive</tt>'s Table of Contents is
at the end of the archive. The executable can open itself as a binary
file name, seek to the end and 'open' the <tt class="docutils literal">CArchive</tt> (see figure 3).</p>
<p>On other platforms, the archive and the executable are separate, but the
archive is named <tt class="docutils literal">executable.pkg</tt>, and expected to be in the same directory.
Other than that, the process is the same.</p>
<div class="section" id="one-pass-execution">
<h3><a class="toc-backref" href="#id47">One Pass Execution</a></h3>
<p>In a single directory deployment (<tt class="docutils literal"><span class="pre">--onedir</span></tt>, which is the default),
all of the binaries are already in the file system. In that case, the
embedding app:</p>
<ul class="simple">
<li>opens the archive</li>
<li>starts Python (on Windows, this is done with dynamic loading so one embedding
app binary can be used with any Python version)</li>
<li>imports all the modules which are at the top level of the archive (basically,
bootstraps the import hooks)</li>
<li>mounts the <tt class="docutils literal">ZlibArchive(s)</tt> in the outer archive</li>
<li>runs all the scripts which are at the top level of the archive</li>
<li>finalizes Python</li>
</ul>
</div>
<div class="section" id="two-pass-execution">
<h3><a class="toc-backref" href="#id48">Two Pass Execution</a></h3>
<p>There are a couple situations which require two passes:</p>
<ul class="simple">
<li>a <tt class="docutils literal"><span class="pre">--onefile</span></tt> deployment (on Windows, the files can't be cleaned
up afterwards because Python does not call <tt class="docutils literal">FreeLibrary</tt>; on other
platforms, Python won't find them if they're extracted in the same
process that uses them)</li>
<li><tt class="docutils literal">LD_LIBRARY_PATH</tt> needs to be set to find the binaries (not
extension modules, but modules the extensions are linked to).</li>
</ul>
<p>The first pass:</p>
<ul class="simple">
<li>opens the archive</li>
<li>extracts all the binaries in the archive (in PyInstaller 2.0,
this is always to a temporary directory).</li>
<li>sets a magic environment variable</li>
<li>sets <tt class="docutils literal">LD_LIBRARY_PATH</tt> (non-Windows)</li>
<li>executes itself as a child process (letting the child use his stdin, stdout
and stderr)</li>
<li>waits for the child to exit (on *nix, the child actually replaces the parent)</li>
<li>cleans up the extracted binaries (so on *nix, this is done by the child)</li>
</ul>
<p>The child process executes as in <a class="reference internal" href="#one-pass-execution">One Pass Execution</a> above (the magic
environment variable is what tells it that this is pass two).</p>
<p><img alt="SE_exeImage" src="images/SE_exe.png" /> figure 3 - Self Extracting Executable</p>
<p>There are, of course, quite a few differences between the Windows and
Unix/Linux versions. The major one is that because all of Python on Windows is
in <tt class="docutils literal">pythonXX.dll</tt>, and dynamic loading is so simple-minded, that one
binary can be use with any version of Python. There's much in common,
though, and that C code can be found in <a class="reference external" href="http://www.pyinstaller.org/browser/trunk/source/common/launch.c?rev=latest">source/common/launch.c</a>.</p>
<p>The Unix/Linux build process (which you need to run just once for any version
of Python) makes use of the config information in your install (if you
installed from RPM, you need the Python-development RPM). It also overrides
<tt class="docutils literal">getpath.c</tt> since we don't want it hunting around the filesystem to build
<tt class="docutils literal">sys.path</tt>.</p>
<p>In both cases, while one <cite>PyInstaller</cite> download can be used with any Python
version, you need to have separate installations for each Python version.</p>
</div>
</div>
</div>
<div class="section" id="pyinstaller-archives">
<h1><a class="toc-backref" href="#id49">PyInstaller Archives</a></h1>
<div class="section" id="archives-introduction">
<h2><a class="toc-backref" href="#id50">Archives Introduction</a></h2>
<p>You know what an archive is: a <tt class="docutils literal">.tar</tt> file, a <tt class="docutils literal">.jar</tt> file, a
<tt class="docutils literal">.zip</tt> file. Two kinds of archives are used here. One is equivalent
to a Java <tt class="docutils literal">.jar</tt> file - it allows Python modules to be stored
efficiently and, (with some import hooks) imported directly. This is a
<tt class="docutils literal">ZlibArchive</tt>. The other (a <tt class="docutils literal">CArchive</tt>) is equivalent to a
<tt class="docutils literal">.zip</tt> file - a general way of packing up (and optionally
compressing) arbitrary blobs of data. It gets its name from the fact
that it can be manipulated easily from C, as well as from Python. Both
of these derive from a common base class, making it fairly easy to
create new kinds of archives.</p>
</div>
<div class="section" id="zlibarchive">
<h2><a class="toc-backref" href="#id51"><tt class="docutils literal">ZlibArchive</tt></a></h2>
<p>A <tt class="docutils literal">ZlibArchive</tt> contains compressed <tt class="docutils literal">.pyc</tt> (or <tt class="docutils literal">.pyo</tt>) files.
The Table of Contents is a marshalled dictionary, with the key (the
module's name as given in an <tt class="docutils literal">import</tt> statement) associated with a
seek position and length. Because it is all marshalled Python,
<tt class="docutils literal">ZlibArchives</tt> are completely cross-platform.</p>
<p>A <tt class="docutils literal">ZlibArchive</tt> hooks in with <a class="reference internal" href="#iu-py">iu.py</a> so that, with a little setup,
the archived modules can be imported transparently. Even with
compression at level 9, this works out to being faster than the normal
import. Instead of searching <tt class="docutils literal">sys.path</tt>, there's a lookup in the
dictionary. There's no <tt class="docutils literal">stat</tt>-ing of the <tt class="docutils literal">.py</tt> and <tt class="docutils literal">.pyc</tt> and no
file opens (the file is already open). There's just a seek, a read and
a decompress. A traceback will point to the source file the archive
entry was created from (the <tt class="docutils literal">__file__</tt> attribute from the time the
<tt class="docutils literal">.pyc</tt> was compiled). On a user's box with no source installed, this
is not terribly useful, but if they send you the traceback, at least
you can make sense of it.</p>
<p><img alt="ZlibArchiveImage" src="images/ZlibArchive.png" /></p>
</div>
<div class="section" id="carchive">
<h2><a class="toc-backref" href="#id52"><tt class="docutils literal">CArchive</tt></a></h2>
<p>A <tt class="docutils literal">CArchive</tt> contains whatever you want to stuff into it. It's very
much like a <tt class="docutils literal">.zip</tt> file. They are easy to create in Python and
unpack from C code. <tt class="docutils literal">CArchives</tt> can be appended to other files (like
ELF and COFF executables, for example). To allow this, they are opened
from the end, so the <tt class="docutils literal">TOC</tt> for a <tt class="docutils literal">CArchive</tt> is at the back,
followed only by a cookie that tells you where the <tt class="docutils literal">TOC</tt> starts and
where the archive itself starts.</p>
<p><tt class="docutils literal">CArchives</tt> can also be embedded within other <tt class="docutils literal">CArchives</tt>. The
inner archive can be opened in place (without extraction).</p>
<p>Each <tt class="docutils literal">TOC</tt> entry is variable length. The first field in the entry
tells you the length of the entry. The last field is the name of the
corresponding packed file. The name is null terminated. Compression is
optional by member.</p>
<p>There is also a type code associated with each entry. If you're using
a <tt class="docutils literal">CArchive</tt> as a <tt class="docutils literal">.zip</tt> file, you don't need to worry about this.
The type codes are used by the self-extracting executables.</p>
<p><img alt="CArchiveImage" src="images/CArchive.png" /></p>
</div>
</div>
<div class="section" id="license">
<h1><a class="toc-backref" href="#id53">License</a></h1>
<p>PyInstaller is mainly distributed under the <a class="reference external" href="http://www.pyinstaller.org/browser/trunk/doc/LICENSE.GPL?rev=latest">GPL License</a> but it has
an exception such that you can use it to compile commercial products.</p>
<p>In a nutshell, the license is GPL for the source code with the exception that:</p>
<blockquote>
<ol class="arabic simple">
<li>You may use PyInstaller to compile commercial applications out of your
source code.</li>
<li>The resulting binaries generated by PyInstaller from your source code can be
shipped with whatever license you want.</li>
<li>You may modify PyInstaller for your own needs but <em>these</em> changes to the
PyInstaller source code falls under the terms of the GPL license. In other
words, any modifications to will <em>have</em> to be distributed under GPL.</li>
</ol>
</blockquote>
<p>For updated information or clarification see our
<a class="reference external" href="http://www.pyinstaller.org/wiki/FAQ">FAQ</a> at <a class="reference external" href="http://www.pyinstaller.org">PyInstaller</a> home page.</p>
</div>
<div class="section" id="appendix">
<h1><a class="toc-backref" href="#id54">Appendix</a></h1>
<div class="sidebar">
<p class="first sidebar-title">You can stop reading here...</p>
<p class="last">... if you are not interested in technical details. This appendix
contains insights of the internal workings of <cite>PyInstaller</cite>, and
you do not need this information unless you plan to work on
<cite>PyInstaller</cite> itself.</p>
</div>
<div class="section" id="building-the-bootloaders">
<h2><a class="toc-backref" href="#id55">Building the bootloaders</a></h2>
<p>PyInstaller comes with binary bootloaders for most platforms, shipped
in /your/path/to/pyinstaller//support/loader. If you need to build the bootloader
for your own platform (either because your platform is not officially
supported, or because you tweaked bootloader's source code), you can
follow this guide.</p>
<div class="section" id="development-tools">
<h3><a class="toc-backref" href="#id56">Development tools</a></h3>
<p>On Debian/Ubuntu systems, you can run the following lines to install everything
required:</p>
<pre class="literal-block">
sudo apt-get install build-essential python-dev
</pre>
<p>On Fedora/RHEL and derivates, you can run the following lines:</p>
<pre class="literal-block">
su
yum groupinstall &quot;Development Tools&quot;
yum install python-devel
</pre>
<p>On Mac OS X you can get gcc by installing <a class="reference external" href="http://developer.apple.com/xcode">Xcode</a>. It is a suite of tools
for developing software for Mac OS X. It can be also installed from your
<tt class="docutils literal">Mac OS X Install DVD</tt>. It is not necessary to install the version 4 of Xcode.</p>
<p>On Solaris and AIX the bootloader is tested with gcc.</p>
<p>On Windows you can use MinGW (gcc for Windows) and Visual Studio C++ (msvc)
compilers. Python development libraries are usually installed together with
Python.</p>
<p><em>Note:</em> There is no interdependence between the Visual Studio
version used to compile the bootloader and the Visual Studio version used to
compile Python. The bootloader is a self-contained static executable,
that imposes no restrictions on the version of Python being used. So
you can simply use any Visual Studio version you have around.</p>
<p>You can download and install or unpack MinGW distribution from one of the
following locations:</p>
<ul class="simple">
<li><a class="reference external" href="http://sourceforge.net/downloads/mingw/">MinGW</a> - stable and mature, uses gcc 3.4 as its base</li>
<li><a class="reference external" href="http://mingw-w64.sourceforge.net/">MinGW-w64</a> - more recent, uses gcc 4.4 and up.</li>
<li><a class="reference external" href="http://tdm-gcc.tdragon.net/">TDM-GCC</a> - MinGW and MinGW-w64 installers</li>
</ul>
</div>
<div class="section" id="building">
<h3><a class="toc-backref" href="#id57">Building</a></h3>
<p>On Windows, when using MinGW, it is needed to add <tt class="docutils literal">PATH_TO_MINGW\bin</tt>
to your system <tt class="docutils literal">PATH</tt>. variable. In command prompt before building
bootloader run for example:</p>
<pre class="literal-block">
set PATH=C:\MinGW\bin;%PATH%
</pre>
<p>Change to the /your/path/to/pyinstaller/ <tt class="docutils literal">source</tt> subdirectory. Run:</p>
<pre class="literal-block">
pyinstaller$ cd source
pyinstaller/source$ python waf configure build install
</pre>
<p>This will produce <tt class="docutils literal">support/loader/YOUR_OS/run</tt>,
<tt class="docutils literal">support/loader/YOUR_OS/run_d</tt>, <tt class="docutils literal">support/loader/YOUR_OS/runw</tt> and
<tt class="docutils literal">support/loader/YOUR_OS/runw_d</tt>, which are the bootloaders.</p>
<p>On Windows this will produce in the <tt class="docutils literal">support/loader/YOUR_OS</tt> directory:
<tt class="docutils literal"><span class="pre">run*.exe</span></tt> (bootloader for regular programs), and
<tt class="docutils literal"><span class="pre">inprocsrvr*.dll</span></tt> (bootloader for in-process COM servers).</p>
<p><em>Note:</em> If you have multiple versions of Python, the Python you use to run
<tt class="docutils literal">waf</tt> is the one whose configuration is used.</p>
<p><em>Note:</em> On AIX the bootloader builds with gcc and is tested with gcc 4.2.0
on AIX 6.1.</p>
</div>
<div class="section" id="linux-standard-base-lsb-binary">
<h3><a class="toc-backref" href="#id58">Linux Standard Base (LSB) binary</a></h3>
<p>By default, the bootloaders on Linux are LSB binaries.</p>
<p>LSB is a set of open standards that should increase compatibility among Linux
distributions. <cite>PyInstaller</cite> is able produce bootloader as LSB binary in order
to increase compatibility for packaged applications among distributions.</p>
<p><em>Note:</em> LSB version 4.0 is required for successfull building of bootloader.</p>
<p>On Debian- and Ubuntu-based distros, you can install LSB 4.0 tools by adding
the following repository to the sources.list file:</p>
<pre class="literal-block">
deb http://ftp.linux-foundation.org/pub/lsb/repositories/debian lsb-4.0 main
</pre>
<p>then after having update the apt repository:</p>
<pre class="literal-block">
sudo apt-get update
</pre>
<p>you can install LSB 4.0:</p>
<pre class="literal-block">
sudo apt-get install lsb lsb-build-cc
</pre>
<p>Most other distributions contain only LSB 3.0 in their software
repositories and thus LSB build tools 4.0 must be downloaded by hand.
From Linux Foundation download <a class="reference external" href="http://ftp.linuxfoundation.org/pub/lsb/bundles/released-4.0.0/sdk/">LSB sdk 4.0</a> for your architecture.</p>
<p>Unpack it by:</p>
<pre class="literal-block">
tar -xvzf lsb-sdk-4.0.3-1.ia32.tar.gz
</pre>
<p>To install it run:</p>
<pre class="literal-block">
cd lsb-sdk
./install.sh
</pre>
<p>After having installed the LSB tools, you can follow the standard building
instructions.</p>
<p><em>NOTE:</em> if for some reason you want to avoid LSB compilation, you can
do so by specifying --no-lsb on the waf command line, as follows:</p>
<pre class="literal-block">
pyinstaller/source$ python waf configure --no-lsb build install
</pre>
<p>This will also produce <tt class="docutils literal">support/loader/YOUR_OS/run</tt>,
<tt class="docutils literal">support/loader/YOUR_OS/run_d</tt>, <tt class="docutils literal">support/loader/YOUR_OS/runw</tt> and
<tt class="docutils literal">support/loader/YOUR_OS/runw_d</tt>, but they will not be LSB binaries.</p>
</div>
</div>
<div class="section" id="mf-py-a-modulefinder-replacement">
<h2><a class="toc-backref" href="#id59"><tt class="docutils literal">mf.py</tt>: A Modulefinder Replacement</a></h2>
<p>Module <tt class="docutils literal">mf</tt> is modelled after <tt class="docutils literal">iu</tt>.</p>
<p>It also uses <tt class="docutils literal">ImportDirectors</tt> and <tt class="docutils literal">Owners</tt> to partition the
import name space. Except for the fact that these return <tt class="docutils literal">Module</tt>
instances instead of real module objects, they are identical.</p>
<p>Instead of an <tt class="docutils literal">ImportManager</tt>, <tt class="docutils literal">mf</tt> has an <tt class="docutils literal">ImportTracker</tt>
managing things.</p>
<div class="section" id="importtracker">
<h3><a class="toc-backref" href="#id60">ImportTracker</a></h3>
<p><tt class="docutils literal">ImportTracker</tt> can be called in two ways: <tt class="docutils literal">analyze_one(name,
importername=None)</tt> or <tt class="docutils literal">analyze_r(name, importername=None)</tt>. The
second method does what modulefinder does - it recursively finds all
the module names that importing name would cause to appear in
<tt class="docutils literal">sys.modules</tt>. The first method is non-recursive. This is useful,
because it is the only way of answering the question &quot;Who imports
name?&quot; But since it is somewhat unrealistic (very few real imports do
not involve recursion), it deserves some explanation.</p>
</div>
<div class="section" id="analyze-one">
<h3><a class="toc-backref" href="#id61"><tt class="docutils literal">analyze_one()</tt></a></h3>
<p>When a name is imported, there are structural and dynamic effects. The dynamic
effects are due to the execution of the top-level code in the module (or
modules) that get imported. The structural effects have to do with whether the
import is relative or absolute, and whether the name is a dotted name (if there
are N dots in the name, then N+1 modules will be imported even without any code
running).</p>
<p>The analyze_one method determines the structural effects, and defers
the dynamic effects. For example, <tt class="docutils literal"><span class="pre">analyze_one(&quot;B.C&quot;,</span> &quot;A&quot;)</tt> could
return <tt class="docutils literal">[&quot;B&quot;, &quot;B.C&quot;]</tt> or <tt class="docutils literal">[&quot;A.B&quot;, &quot;A.B.C&quot;]</tt> depending on whether
the import turns out to be relative or absolute. In addition,
ImportTracker's modules dict will have Module instances for them.</p>
</div>
<div class="section" id="module-classes">
<h3><a class="toc-backref" href="#id62">Module Classes</a></h3>
<p>There are Module subclasses for builtins, extensions, packages and (normal)
modules. Besides the normal module object attributes, they have an attribute
imports. For packages and normal modules, imports is a list populated by
scanning the code object (and therefor, the names in this list may be relative
or absolute names - we don't know until they have been analyzed).</p>
<p>The highly astute will notice that there is a hole in
<tt class="docutils literal">analyze_one()</tt> here. The first thing that happens when <tt class="docutils literal">B.C</tt> is
being imported is that <tt class="docutils literal">B</tt> is imported and it's top-level code
executed. That top-level code can do various things so that when the
import of <tt class="docutils literal">B.C</tt> finally occurs, something completely different
happens (from what a structural analysis would predict). But mf can
handle this through it's hooks mechanism.</p>
</div>
<div class="section" id="code-scanning">
<h3><a class="toc-backref" href="#id63">code scanning</a></h3>
<p>Like modulefinder, <tt class="docutils literal">mf</tt> scans the byte code of a module, looking for
imports. In addition, <tt class="docutils literal">mf</tt> will pick out a module's <tt class="docutils literal">__all__</tt>
attribute, if it is built as a list of constant names. This means that
if a package declares an <tt class="docutils literal">__all__</tt> list as a list of names,
ImportTracker will track those names if asked to analyze
<tt class="docutils literal">package.*</tt>. The code scan also notes the occurance of
<tt class="docutils literal">__import__</tt>, <tt class="docutils literal">exec</tt> and <tt class="docutils literal">eval</tt>, and can issue warnings when
they're found.</p>
<p>The code scanning also keeps track (as well as it can) of the context of an
import. It recognizes when imports are found at the top-level, and when they
are found inside definitions (deferred imports). Within that, it also tracks
whether the import is inside a condition (conditional imports).</p>
</div>
<div class="section" id="hooks">
<h3><a class="toc-backref" href="#id64">Hooks</a></h3>
<p>In modulefinder, scanning the code takes the place of executing the
code object. <tt class="docutils literal">ExtensionModules</tt>, of course, don't get scanned, so
there need to be a way of recording any imports they do.</p>
<p>Please read <a class="reference internal" href="#listing-hidden-imports">Listing Hidden Imports</a> for more information.</p>
<p><tt class="docutils literal">mf</tt> goes further and allows a module to be hooked (after it has been
scanned, but before analyze_one is done with it). A hook is a module named
<tt class="docutils literal"><span class="pre">hook-fully.qualified.name</span></tt> in the <tt class="docutils literal">PyInstaller.hooks</tt> package.</p>
<p>These modules should have one or more of the following three global
names defined:</p>
<dl class="docutils">
<dt><tt class="docutils literal">hiddenimports</tt></dt>
<dd><p class="first">A list of modules names (relative or absolute) the
module imports in some untrackable way.</p>
<p>This extends the list of modules to be imported which is created
by scanning the code.</p>
<p>Example:</p>
<pre class="last literal-block">
hiddenimports = ['_proxy', 'utils', 'defs']
</pre>
</dd>
<dt><tt class="docutils literal">datas</tt></dt>
<dd><p class="first">A list of globs of files or directories to bundle as datafiles. For
each glob, a destination directory is specified.</p>
<p>Example:</p>
<pre class="literal-block">
datas = [
     ('/usr/share/icons/education_*.png', 'icons'),
     ('/usr/share/libsmi/mibs/*', 'mibs'),
     ]
</pre>
<p class="last">This will copy all iconfiles matching <cite>education_*.png</cite> into the
subdirectory <cite>icons</cite> and recursively copy the content of
<cite>/usr/share/libsmi/mibs</cite> into <cite>mibs</cite>.</p>
</dd>
<dt><tt class="docutils literal">attrs</tt></dt>
<dd><p class="first">A list of <tt class="docutils literal">(name, value)</tt> pairs (where value is normally
meaningless).</p>
<p>This will set the module-attribute <tt class="docutils literal">name</tt> to <tt class="docutils literal">value</tt> for each
pait in the list. The value is meaningless normally, since the
modules are not executed.</p>
<p>This exists mainly so that ImportTracker won't issue spurious
warnings when the rightmost node in a dotted name turns out to be
an attribute in a package, instead of a missing submodule.</p>
<p class="last">Example: See <tt class="docutils literal"><span class="pre">PyInstaller/hooks/hook-xml.dom.ext.py</span></tt>.</p>
</dd>
<dt><tt class="docutils literal">hook(mod)</tt></dt>
<dd><p class="first">A function expecting a <tt class="docutils literal">Module</tt> instance and
returning a <tt class="docutils literal">Module</tt> instance (so it can modify or replace).</p>
<p>This exists for things like dynamic modification of a
package's <tt class="docutils literal">__path__</tt> or perverse situations, like
<tt class="docutils literal">xml.__init__</tt> replacing itself in <tt class="docutils literal">sys.modules</tt> with
<tt class="docutils literal">_xmlplus.__init__</tt>. (It takes nine hook modules to properly
trace through PyXML-using code, and I can't believe that it's any
easier for the poor programmer using that package).</p>
<p class="last">The <tt class="docutils literal">hook(mod)</tt> (if it exists) is called before looking at the
others - that way it can, for example, test <tt class="docutils literal">sys.version</tt> and
adjust what's in <tt class="docutils literal">hiddenimports</tt>.</p>
</dd>
</dl>
</div>
</div>
<div class="section" id="advanced-hook-usage">
<h2><a class="toc-backref" href="#id65">Advanced Hook Usage</a></h2>
<p>Since the hook module is imported like any other
module, you can use any Python code we need. For example for
colletiong additional data or files. See the existing hooks in
<tt class="docutils literal">PyInstaller/hooks</tt> for some examples, esp. the <tt class="docutils literal">django</tt> hooks.</p>
<div class="section" id="warnings">
<h3><a class="toc-backref" href="#id66">Warnings</a></h3>
<p><tt class="docutils literal">ImportTracker</tt> has a <tt class="docutils literal">getwarnings()</tt> method that returns all the
warnings accumulated by the instance, and by the <tt class="docutils literal">Module</tt> instances
in its modules dict. Generally, it is <tt class="docutils literal">ImportTracker</tt> who will
accumulate the warnings generated during the structural phase, and
<tt class="docutils literal">Modules</tt> that will get the warnings generated during the code scan.</p>
<p>Note that by using a hook module, you can silence some particularly tiresome
warnings, but not all of them.</p>
</div>
<div class="section" id="cross-reference">
<h3><a class="toc-backref" href="#id67">Cross Reference</a></h3>
<p>Once a full analysis (that is, an <tt class="docutils literal">analyze_r</tt> call) has been done,
you can get a cross reference by using <tt class="docutils literal">getxref()</tt>. This returns a
list of tuples. Each tuple is <tt class="docutils literal">(modulename, importers)</tt>, where
importers is a list of the (fully qualified) names of the modules
importing <tt class="docutils literal">modulename</tt>. Both the returned list and the importers
list are sorted.</p>
</div>
<div class="section" id="mf-usage">
<h3><a class="toc-backref" href="#id68">mf Usage</a></h3>
<p>A simple example follows:</p>
<blockquote>
<pre class="doctest-block">
&gt;&gt;&gt; import mf
&gt;&gt;&gt; a = mf.ImportTracker()
&gt;&gt;&gt; a.analyze_r(&quot;os&quot;)
['os', 'sys', 'posixpath', 'nt', 'stat', 'string', 'strop',
're', 'pcre', 'ntpath', 'dospath', 'macpath', 'win32api',
'UserDict', 'copy', 'types', 'repr', 'tempfile']
&gt;&gt;&gt; a.analyze_one(&quot;os&quot;)
['os']
&gt;&gt;&gt; a.modules['string'].imports
[('strop', 0, 0), ('strop.*', 0, 0), ('re', 1, 1)]
&gt;&gt;&gt;
</pre>
</blockquote>
<p>The tuples in the imports list are (name, delayed, conditional).</p>
<blockquote>
<pre class="doctest-block">
&gt;&gt;&gt; for w in a.modules['string'].warnings: print w
...
W: delayed  eval hack detected at line 359
W: delayed  eval hack detected at line 389
W: delayed  eval hack detected at line 418
&gt;&gt;&gt; for w in a.getwarnings(): print w
...
W: no module named pwd (delayed, conditional import by posixpath)
W: no module named dos (conditional import by os)
W: no module named os2 (conditional import by os)
W: no module named posix (conditional import by os)
W: no module named mac (conditional import by os)
W: no module named MACFS (delayed, conditional import by tempfile)
W: no module named macfs (delayed, conditional import by tempfile)
W: top-level conditional exec statment detected at line 47
   - os (C:\Program Files\Python\Lib\os.py)
W: delayed  eval hack detected at line 359
   - string (C:\Program Files\Python\Lib\string.py)
W: delayed  eval hack detected at line 389
   - string (C:\Program Files\Python\Lib\string.py)
W: delayed  eval hack detected at line 418
   - string (C:\Program Files\Python\Lib\string.py)
&gt;&gt;&gt;
</pre>
</blockquote>
</div>
</div>
<div class="section" id="iu-py-an-imputil-replacement">
<span id="iu-py"></span><h2><a class="toc-backref" href="#id69"><tt class="docutils literal">iu.py</tt>: An <em>imputil</em> Replacement</a></h2>
<p>Module <tt class="docutils literal">iu</tt> grows out of the pioneering work that Greg Stein did
with <tt class="docutils literal">imputil</tt> (actually, it includes some verbatim <tt class="docutils literal">imputil</tt>
code, but since Greg didn't copyright it, we won't mention it). Both
modules can take over Python's builtin import and ease writing of at
least certain kinds of import hooks.</p>
<p><tt class="docutils literal">iu</tt> differs from <tt class="docutils literal">imputil</tt>:
* faster
* better emulation of builtin import
* more managable</p>
<p>There is an <tt class="docutils literal">ImportManager</tt> which provides the replacement for builtin import
and hides all the semantic complexities of a Python import request from it's
delegates.</p>
<div class="section" id="importmanager">
<h3><a class="toc-backref" href="#id70"><tt class="docutils literal">ImportManager</tt></a></h3>
<p><tt class="docutils literal">ImportManager</tt> formalizes the concept of a metapath. This concept implicitly
exists in native Python in that builtins and frozen modules are searched
before <tt class="docutils literal">sys.path</tt>, (on Windows there's also a search of the registry while on
Mac, resources may be searched). This metapath is a list populated with
<tt class="docutils literal">ImportDirector</tt> instances. There are <tt class="docutils literal">ImportDirector</tt> subclasses
for builtins, frozen modules, (on Windows) modules found through the
registry and a <tt class="docutils literal">PathImportDirector</tt> for handling <tt class="docutils literal">sys.path</tt>. For a
top-level import (that is, not an import of a module in a package),
<tt class="docutils literal">ImportManager</tt> tries each director on it's metapath until one
succeeds.</p>
<p><tt class="docutils literal">ImportManager</tt> hides the semantic complexity of an import from the directors.
It's up to the <tt class="docutils literal">ImportManager</tt> to decide if an import is relative or absolute;
to see if the module has already been imported; to keep <tt class="docutils literal">sys.modules</tt> up to
date; to handle the fromlist and return the correct module object.</p>
</div>
<div class="section" id="importdirector">
<h3><a class="toc-backref" href="#id71"><tt class="docutils literal">ImportDirector</tt></a></h3>
<p>An <tt class="docutils literal">ImportDirector</tt> just needs to respond to <tt class="docutils literal">getmod(name)</tt> by
returning a module object or <tt class="docutils literal">None</tt>. As you will see, an
<tt class="docutils literal">ImportDirector</tt> can consider name to be atomic - it has no need to
examine name to see if it is dotted.</p>
<p>To see how this works, we need to examine the <tt class="docutils literal">PathImportDirector</tt>.</p>
</div>
<div class="section" id="pathimportdirector">
<h3><a class="toc-backref" href="#id72"><tt class="docutils literal">PathImportDirector</tt></a></h3>
<p>The <tt class="docutils literal">PathImportDirector</tt> subclass manages a list of names - most
notably, <tt class="docutils literal">sys.path</tt>. To do so, it maintains a shadowpath - a
dictionary mapping the names on its pathlist (eg, <tt class="docutils literal">sys.path</tt>) to
their associated <tt class="docutils literal">Owners</tt>. (It could do this directly, but the
assumption that sys.path is occupied solely by strings seems
ineradicable.) <tt class="docutils literal">Owners</tt> of the appropriate kind are created as
needed (if all your imports are satisfied by the first two elements of
<tt class="docutils literal">sys.path</tt>, the <tt class="docutils literal">PathImportDirector</tt>'s shadowpath will only have
two entries).</p>
</div>
<div class="section" id="owner">
<h3><a class="toc-backref" href="#id73"><tt class="docutils literal">Owner</tt></a></h3>
<p>An <tt class="docutils literal">Owner</tt> is much like an <tt class="docutils literal">ImportDirector</tt> but manages a much
more concrete piece of turf. For example, a <tt class="docutils literal">DirOwner</tt> manages one
directory. Since there are no other officially recognized
filesystem-like namespaces for importing, that's all that's included
in iu, but it's easy to imagine <tt class="docutils literal">Owners</tt> for zip files (and I have
one for my own <tt class="docutils literal">.pyz</tt> archive format) or even URLs.</p>
<p>As with <tt class="docutils literal">ImportDirectors</tt>, an <tt class="docutils literal">Owner</tt> just needs to respond to
<tt class="docutils literal">getmod(name)</tt> by returning a module object or <tt class="docutils literal">None</tt>, and it can
consider name to be atomic.</p>
<p>So structurally, we have a tree, rooted at the <tt class="docutils literal">ImportManager</tt>. At
the next level, we have a set of <tt class="docutils literal">ImportDirectors</tt>. At least one of
those directors, the <tt class="docutils literal">PathImportDirector</tt> in charge of <tt class="docutils literal">sys.path</tt>,
has another level beneath it, consisting of <tt class="docutils literal">Owners</tt>. This much of
the tree covers the entire top-level import namespace.</p>
<p>The rest of the import namespace is covered by treelets, each rooted in a
package module (an <tt class="docutils literal">__init__.py</tt>).</p>
</div>
<div class="section" id="packages">
<h3><a class="toc-backref" href="#id74">Packages</a></h3>
<p>To make this work, <tt class="docutils literal">Owners</tt> need to recognize when a module is a
package. For a <tt class="docutils literal">DirOwner</tt>, this means that name is a subdirectory
which contains an <tt class="docutils literal">__init__.py</tt>. The <tt class="docutils literal">__init__</tt> module is loaded
and its <tt class="docutils literal">__path__</tt> is initialized with the subdirectory. Then, a
<tt class="docutils literal">PathImportDirector</tt> is created to manage this <tt class="docutils literal">__path__</tt>. Finally
the new <tt class="docutils literal">PathImportDirector</tt>'s <tt class="docutils literal">getmod</tt> is assigned to the
package's <tt class="docutils literal">__importsub__</tt> function.</p>
<p>When a module within the package is imported, the request is routed
(by the <tt class="docutils literal">ImportManager</tt>) diretly to the package's <tt class="docutils literal">__importsub__</tt>.
In a hierarchical namespace (like a filesystem), this means that
<tt class="docutils literal">__importsub__</tt> (which is really the bound getmod method of a
<tt class="docutils literal">PathImportDirector</tt> instance) needs only the module name, not the
package name or the fully qualified name. And that's exactly what it
gets. (In a flat namespace - like most archives - it is perfectly easy
to route the request back up the package tree to the archive
<tt class="docutils literal">Owner</tt>, qualifying the name at each step.)</p>
</div>
<div class="section" id="possibilities">
<h3><a class="toc-backref" href="#id75">Possibilities</a></h3>
<p>Let's say we want to import from zip files. So, we subclass <tt class="docutils literal">Owner</tt>.
The <tt class="docutils literal">__init__</tt> method should take a filename, and raise a
<tt class="docutils literal">ValueError</tt> if the file is not an acceptable <tt class="docutils literal">.zip</tt> file, (when a
new name is encountered on <tt class="docutils literal">sys.path</tt> or a package's <tt class="docutils literal">__path__</tt>,
registered Owners are tried until one accepts the name). The
<tt class="docutils literal">getmod</tt> method would check the zip file's contents and return
<tt class="docutils literal">None</tt> if the name is not found. Otherwise, it would extract the
marshalled code object from the zip, create a new module object and
perform a bit of initialization (12 lines of code all told for my own
archive format, including initializing a pack age with it's
<tt class="docutils literal">__subimporter__</tt>).</p>
<p>Once the new <tt class="docutils literal">Owner</tt> class is registered with <tt class="docutils literal">iu</tt>, you can put a
zip file on <tt class="docutils literal">sys.path</tt>. A package could even put a zip file on its
<tt class="docutils literal">__path__</tt>.</p>
</div>
<div class="section" id="compatibility">
<h3><a class="toc-backref" href="#id76">Compatibility</a></h3>
<p>This code has been tested with the PyXML, mxBase and Win32 packages,
covering over a dozen import hacks from manipulations of <tt class="docutils literal">__path__</tt>
to replacing a module in <tt class="docutils literal">sys.modules</tt> with a different one.
Emulation of Python's native import is nearly exact, including the
names recorded in <tt class="docutils literal">sys.modules</tt> and module attributes (packages
imported through <tt class="docutils literal">iu</tt> have an extra attribute - <tt class="docutils literal">__importsub__</tt>).</p>
</div>
<div class="section" id="performance">
<h3><a class="toc-backref" href="#id77">Performance</a></h3>
<p>In most cases, <tt class="docutils literal">iu</tt> is slower than builtin import (by 15 to 20%) but
faster than <tt class="docutils literal">imputil</tt> (by 15 to 20%). By inserting archives at the
front of <tt class="docutils literal">sys.path</tt> containing the standard lib and the package
being tested, this can be reduced to 5 to 10% slower (or, on my 1.52
box, 10% faster!) than builtin import. A bit more can be shaved off by
manipulating the <tt class="docutils literal">ImportManager</tt>'s metapath.</p>
</div>
<div class="section" id="limitations">
<h3><a class="toc-backref" href="#id78">Limitations</a></h3>
<p>This module makes no attempt to facilitate policy import hacks. It is easy to
implement certain kinds of policies within a particular domain, but
fundamentally iu works by dividing up the import namespace into independent
domains.</p>
<p>Quite simply, I think cross-domain import hacks are a very bad idea. As author
of the original package on which <cite>PyInstaller</cite> is based, McMillan worked with
import hacks for many years. Many of them are highly fragile; they often rely
on undocumented (maybe even accidental) features of implementation.
A cross-domain import hack is not likely to work with PyXML, for example.</p>
<p>That rant aside, you can modify <tt class="docutils literal">ImportManger</tt> to implement
different policies. For example, a version that implements three
import primitives: absolute import, relative import and
recursive-relative import. No idea what the Python syntax for those
should be, but <tt class="docutils literal">__aimport__</tt>, <tt class="docutils literal">__rimport__</tt> and <tt class="docutils literal">__rrimport__</tt>
were easy to implement.</p>
</div>
<div class="section" id="iu-usage">
<h3><a class="toc-backref" href="#id79">iu Usage</a></h3>
<p>Here's a simple example of using <tt class="docutils literal">iu</tt> as a builtin import replacement.</p>
<blockquote>
<pre class="doctest-block">
&gt;&gt;&gt; import iu
&gt;&gt;&gt; iu.ImportManager().install()
&gt;&gt;&gt;
&gt;&gt;&gt; import DateTime
&gt;&gt;&gt; DateTime.__importsub__
&lt;method PathImportDirector.getmod
  of PathImportDirector instance at 825900&gt;
&gt;&gt;&gt;
</pre>
</blockquote>
</div>
</div>
</div>
</div>
<div class="footer">
<hr class="footer" />
<a class="reference external" href="source/Manual.rst">View document source</a>.
Generated on: 2012-08-08 18:25 UTC.
Generated by <a class="reference external" href="http://docutils.sourceforge.net/">Docutils</a> from <a class="reference external" href="http://docutils.sourceforge.net/rst.html">reStructuredText</a> source.

</div>
</body>
</html>
